-- Logs begin at Thu 2022-07-14 18:06:24 UTC, end at Thu 2022-07-14 19:33:57 UTC. -- Jul 14 18:47:58.924598 dv-cinder-799871-31-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 14 18:47:58.934665 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 14 18:47:58.935202 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 14 18:47:58 2022] *** Jul 14 18:47:58.935202 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 14 18:47:58.935202 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 14 18:47:58.935202 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: nodename: dv-cinder-799871-31-iscsi Jul 14 18:47:58.935202 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: machine: x86_64 Jul 14 18:47:58.935202 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: clock source: unix Jul 14 18:47:58.935202 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: pcre jit disabled Jul 14 18:47:58.935202 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: detected number of CPU cores: 4 Jul 14 18:47:58.935202 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: current working directory: / Jul 14 18:47:58.935202 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: detected binary path: /usr/bin/uwsgi-core Jul 14 18:47:58.935202 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: your processes number limit is 47680 Jul 14 18:47:58.935202 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: your memory page size is 4096 bytes Jul 14 18:47:58.935202 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: detected max file descriptor number: 2048 Jul 14 18:47:58.935202 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: lock engine: pthread robust mutexes Jul 14 18:47:58.935879 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: thunder lock: enabled Jul 14 18:47:58.935879 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 14 18:47:58.935879 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 14 18:47:58.977155 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: Python main interpreter initialized at 0x5563342439c0 Jul 14 18:47:58.977155 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: python threads support enabled Jul 14 18:47:58.977155 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: your server socket listen backlog is limited to 100 connections Jul 14 18:47:58.977155 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: your mercy for graceful operations on workers is 90 seconds Jul 14 18:47:58.977528 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: mapped 403077 bytes (393 KB) for 2 cores Jul 14 18:47:58.977604 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: *** Operational MODE: preforking *** Jul 14 18:47:58.977840 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: *** uWSGI is running in multiple interpreter mode *** Jul 14 18:47:58.977840 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: spawned uWSGI master process (pid: 80328) Jul 14 18:47:58.978046 dv-cinder-799871-31-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 14 18:47:58.979197 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: spawned uWSGI worker 1 (pid: 80342, cores: 1) Jul 14 18:47:58.979197 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: spawned uWSGI worker 2 (pid: 80343, cores: 1) Jul 14 18:47:58.979197 dv-cinder-799871-31-iscsi devstack@keystone.service[80328]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 14 18:48:04.409701 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 18:48:04.409701 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: from cryptography.utils import int_from_bytes Jul 14 18:48:04.410958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 18:48:04.410958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: from cryptography.utils import int_from_bytes Jul 14 18:48:04.411396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 18:48:04.411396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: from cryptography.utils import int_from_bytes Jul 14 18:48:04.411502 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 18:48:04.411502 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: from cryptography.utils import int_from_bytes Jul 14 18:48:04.459688 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 18:48:04.469075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 18:48:04.469165 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 18:48:04.469258 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 18:48:04.469332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] command line args: [] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 18:48:04.469391 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] command line args: [] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 18:48:04.469447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 18:48:04.469507 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 18:48:04.469563 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ================================================================================ {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 18:48:04.469624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ================================================================================ {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 18:48:04.469691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] admin_token = **** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.469754 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] admin_token = **** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.469815 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] config_dir = [] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.469877 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] config_dir = [] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.469938 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.470001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.470066 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] config_source = [] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.470128 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] config_source = [] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.470190 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] debug = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.470253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] debug = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.470310 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.470615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.470615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.470901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.471021 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.471132 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.471279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.471399 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.471513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.471629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.471629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] list_limit = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.471850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] list_limit = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.471970 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] log_config_append = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.472110 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] log_config_append = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.472237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.472357 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.473140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] log_dir = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.473262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] log_dir = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.473391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] log_file = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.473519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] log_file = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.473644 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.473889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.474018 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.474147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.474279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.482797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.482894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.482977 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.483039 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.483174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.483265 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.483338 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.483413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.483483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.483552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.483631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.483702 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.483770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.483838 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.483912 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.483981 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.484059 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.484134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.484209 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.484279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.484360 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.484435 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.484514 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.484584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.484658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.484733 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.69/identity {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.484812 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.69/identity {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.484888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] publish_errors = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.484958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] publish_errors = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.485031 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.485166 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.485239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.485312 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.485381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.485454 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.485523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.485608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.485688 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.485759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.485833 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] standard_threads = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.485910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] standard_threads = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.485980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.486053 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.486122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.486190 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.486263 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.486339 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.486409 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] use_journal = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.486477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] use_journal = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.486545 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] use_json = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.486618 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] use_json = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.486692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] use_stderr = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.486769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] use_stderr = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.486844 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] use_syslog = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.486913 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] use_syslog = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.486987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.487062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 18:48:04.487131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.487305 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.487379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.487448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.487515 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.487589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.487664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.487744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.487869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.487943 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.488012 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.488086 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.488161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.488238 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.488308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.488376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.488444 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.488596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.488672 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.488672 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.488672 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.488672 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.488915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.488983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.489057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.489134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.489209 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.489279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.489354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.489430 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.489505 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.489583 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.489658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.489728 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.489796 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.489863 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.489942 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.490012 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.490085 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.490155 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.490268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.490341 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.490418 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.490494 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.490565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.490690 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.490778 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.490851 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.491013 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.491090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.491090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.491301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.491397 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.491469 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.491548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.491625 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.491695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.491764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.491838 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.491908 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.491984 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.492060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.492131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.492199 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.492267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.492335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.492411 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.492524 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.492627 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.492729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.492831 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.492931 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.493033 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.493128 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.493221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.493321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.493415 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.493516 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.493621 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.493700 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.493821 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.493927 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.494008 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.494091 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.494170 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.494242 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.494358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.494432 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.494547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.494660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.494822 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.494956 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.495030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.495152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.495261 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.495391 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.495470 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.495600 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.495728 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.496077 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.496150 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.496281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.496378 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.496465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.496589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.496589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.496773 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.496849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.496963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.497047 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.497125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.497197 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.497317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.497393 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.497464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.497588 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.497664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.497741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.497819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.497933 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.498016 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.498097 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.498176 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.498248 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.498318 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.498433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.498433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.498572 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.498645 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.498716 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.498786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.498856 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.498945 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.499029 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.499101 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.499198 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.499272 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.499342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.499412 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.499543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.499621 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.499697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.499776 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.499849 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.499919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.500068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.500142 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.500213 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.500284 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.500354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.500430 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.500510 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.500582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.500656 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.500727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.500727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.500922 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.500997 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.501078 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.501151 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.501221 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.501291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.501361 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.501431 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.501509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.501588 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.501665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.501738 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.501808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.501877 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.501955 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.502028 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.502097 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.502167 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.502242 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.502314 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.502314 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.502458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.502531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.502601 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.502671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.502741 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.502825 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.502899 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.502970 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.503039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.503109 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.503257 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.503345 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.503467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.503542 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.503613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.503683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.503801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.503888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.503962 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.504032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.504102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.504178 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.504250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.504328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] auth.external = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.504401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.504471 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.504541 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.504617 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.504690 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.504769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.504841 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.504916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] auth.password = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.504996 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.505069 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] auth.token = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.505139 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.505217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.505217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.505386 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.505459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.505536 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.505536 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] auth.external = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.505665 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.505735 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.505735 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.505873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.505950 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.506022 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.506092 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] auth.password = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.506092 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] auth.token = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.506228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.506301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.506301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.506428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] credential.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.506498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.506574 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.506654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.506654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.506847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.506938 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.506938 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.507145 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.507256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] credential.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.507332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.507404 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.507404 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.507530 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.507602 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.507602 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.507739 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.507811 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.507811 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.507937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.508013 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.508094 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.508167 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.508167 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.508293 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.508365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.508365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.508544 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.508623 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.508623 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.508757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.508830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.508982 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.509057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.509057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] federation.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.509185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.509255 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.509255 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.509388 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.509461 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.509531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.509601 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.509671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] federation.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.509742 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.509820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.509893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.509963 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.510040 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.510040 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.510170 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.510249 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.510327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.510400 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.510400 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.510526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.510597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.510597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.510737 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.510809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.510809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.510935 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.511006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.511084 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.511146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.511343 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.511417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.511487 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.511557 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.511642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.511716 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.511776 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.511845 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.511915 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.511986 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.512065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.512139 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.512211 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.512281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.512352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.512423 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.512501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.512575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.512646 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.512716 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.512786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.512856 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.512935 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.513008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.513078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.513078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.513205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.513275 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.513354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.513427 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.513497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.513568 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.513638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.513708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.513787 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.513787 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.513927 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.513998 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.514068 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.514139 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.514220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.514358 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.514441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.514610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.514690 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.514843 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.514924 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.515074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.515151 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.515336 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.515416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.515576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.515656 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.515811 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.515890 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.516054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.516134 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.516286 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.516371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.516531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.516610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.516762 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.516842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.517001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.517080 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.517240 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.517320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.517472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.517551 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.517714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.517794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.517954 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.518033 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.518183 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.518261 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.518427 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.518509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.518658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.518737 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.518887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.518970 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.519124 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.519249 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.519932 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.520059 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.520195 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.520306 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.520436 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.520546 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.520686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.520765 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.520935 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.521015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.521168 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.521248 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.521399 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.521484 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.521682 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.521787 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.522032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.522210 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.522287 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.522411 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.522603 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.522683 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.522884 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.523000 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.523199 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.523321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.523469 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.523573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.523710 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.523841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.523949 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.524079 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.524182 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.524331 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.524419 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.524581 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.524661 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.524831 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.524911 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.525069 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.525158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.525302 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.525382 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.525794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.525881 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.526031 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.526120 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.526267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.526347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.526498 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.526577 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.526746 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.526834 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.526982 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.527061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.527252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.527333 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.527500 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.527593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.527737 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.527816 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.527966 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.528043 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.528210 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.528298 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.528446 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.528525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.528681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.528771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.528914 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.529001 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.529145 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.529223 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.529393 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.529472 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.529634 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.529788 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.529864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.529941 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.530093 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.530172 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.530327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.530414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] resource.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.530571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.530651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.530807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.530887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.531043 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.531129 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.531306 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.531386 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.531554 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.531634 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.531791 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.531870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.532023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.532101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.532260 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.532339 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.532500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.532579 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] resource.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.532738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] role.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.532817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.532980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.533058 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] role.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.533211 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.533290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.533456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.533638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.533711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.533780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.533927 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.534007 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.534172 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.534251 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.534408 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.534487 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.534647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.534727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.534880 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] role.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.534966 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.535718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.535805 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] role.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.535952 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.536033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.536202 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.536280 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.536449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.536528 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.536694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.536777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.536940 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.537018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.537186 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.537383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.537471 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.537626 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.537706 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.537882 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.537962 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.538118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.538204 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.538365 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.538444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.538609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.538689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.538846 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.538932 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.539089 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.539185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.539370 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.539552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.539644 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.539788 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.539868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.540038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.540117 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.540280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.540367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.540525 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.540604 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.540772 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.540852 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.541006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.541094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.541254 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.541332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.541494 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.541580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.541728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.541816 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.541958 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.542036 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] token.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.542191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.542270 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.542421 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.542509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.542663 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.542741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.542913 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.542991 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.543178 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.543273 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.543420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.543499 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.543658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.543737 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.543889 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.543977 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.544124 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] token.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.544203 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.544359 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.544439 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.544605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.544735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.544820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.544895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.545076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.545157 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.545320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.545410 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.545552 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.545639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.545797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.545876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.546040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.546131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.546289 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.546376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.connection = **** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.546524 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.546615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.546763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.546845 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.546998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.547081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.547269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.547355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.547517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.547596 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.547747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.547829 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.547982 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.548061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.548221 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.548301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.548469 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.548558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.connection = **** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.548709 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.548799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.548955 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.549035 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.549192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.549273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.549428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.549504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.549664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.549753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.549900 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.549982 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.550131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.550212 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.550467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.550547 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.550743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.550824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.550983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.551066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 18:48:04.551253 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.551491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.551739 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.551976 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.552220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.552452 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.552703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.552941 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 18:48:04.553111 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 18:48:04.585267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.585488 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.585714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.589185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.589422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.589650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.594197 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.598553 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.622809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.623041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.624209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.624440 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.765548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.766141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.814793 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.815026 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.816422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.816660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.882208 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.882395 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.882474 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.882576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.882655 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.882784 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.882869 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.883003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.883096 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.883246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.883331 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.883464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.909157 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.909391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.909500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.909619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.909704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.909845 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.909930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.910112 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.910197 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.910328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.910439 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.910541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.910665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.910888 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.921544 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.922008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.934448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.934682 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.934790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.934906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.934990 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.935129 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.935229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.935371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.935456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.935587 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.935673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.935904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.936013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.936141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 18:48:04.953482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.953692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 18:48:04.953776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.953982 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 18:48:04.954079 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.954318 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.954445 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 18:48:04.954528 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 18:48:04.954719 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.954979 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.955061 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.955268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.955353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.955525 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.955605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.955856 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.967007 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.967303 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.967543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.967626 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.967841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.967926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.968116 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.968396 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.976104 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:04.976670 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:04.979398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:04.979674 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.979953 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.980082 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:04.980237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.980377 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.980508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.980639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.980776 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.980907 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.981046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.981176 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.981305 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.981444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.981573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.981711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.981841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.981971 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.982100 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.982229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.982358 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:04.982570 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.009296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.009636 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.010920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.011223 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.011310 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.011562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.016523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.016752 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.016854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.017031 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.017114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.017351 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.025086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.025353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.025451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.025889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.030085 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.030371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.030451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.030650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.030734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.030928 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.031010 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.031226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.031324 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.031502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.031584 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.031772 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.031853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.032038 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.032128 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.032305 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.050731 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.050919 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.051004 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.051213 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.055545 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.055733 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.059571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.059670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.059872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.059959 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.064983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.065083 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.069138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.069255 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.072743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.072834 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.073038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.073123 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.073321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.073417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.073602 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.073683 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.073896 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.073985 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.074181 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.074264 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.093798 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.093895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.094078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.094161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.094343 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.094425 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.094604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.094694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.102512 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.102621 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.104242 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.104326 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.104517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.104610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.107687 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.107771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.107980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.108062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.108258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.108340 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.108545 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.108624 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.108828 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.108916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.122078 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.122169 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.122355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.122438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.122635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.122717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.122929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.123046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.123214 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.123297 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.123489 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.123573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.123760 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.123839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.143452 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.143564 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.147293 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.147389 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.150856 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.150947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.152485 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.152574 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.154249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.154329 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.154527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.154620 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.154805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.154885 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.162128 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.162244 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.163750 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.163858 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.167571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.167718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.167840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.167994 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.168099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.168199 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.168298 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.168396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.177081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.177193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.177364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.177450 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.177636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.177717 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.185534 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.185806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.186082 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.186211 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 18:48:05.186343 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.186481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.186607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.186753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.186885 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.187015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.187146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.187315 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.187447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.187578 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.187759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.187887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.188024 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.188155 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.188312 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.188439 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.188569 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.188716 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.188847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.188975 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.189104 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.189233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.189478 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.189745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 18:48:05.227670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5563342439c0 pid: 80343 (default app) Jul 14 18:48:05.228429 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5563342439c0 pid: 80342 (default app) Jul 14 18:48:05.231043 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b87c5b-d155-4068-b953-950a7f84bd59 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:05.231363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b87c5b-d155-4068-b953-950a7f84bd59 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:05.231645 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b87c5b-d155-4068-b953-950a7f84bd59 None None] PATH_INFO: `/v3` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:05.235800 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1/1] 10.222.0.69 () {50 vars in 791 bytes} [Thu Jul 14 18:48:05 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:13.642595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f48995-33f9-48a3-be71-b87980023ccb None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:13.643112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f48995-33f9-48a3-be71-b87980023ccb None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:13.643240 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f48995-33f9-48a3-be71-b87980023ccb None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:13.648078 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1/2] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:13.654039 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6574a67-a6de-4d30-8f41-c315ec03f4d2 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:13.654447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6574a67-a6de-4d30-8f41-c315ec03f4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:13.654669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6574a67-a6de-4d30-8f41-c315ec03f4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:13.722633 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: INFO dbcounter [None req-f6574a67-a6de-4d30-8f41-c315ec03f4d2 None None] Registered counter for database keystone Jul 14 18:48:13.745093 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG oslo_db.sqlalchemy.engines [None req-f6574a67-a6de-4d30-8f41-c315ec03f4d2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80343) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 18:48:13.751334 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writer thread running {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 18:48:13.931642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG passlib.handlers.bcrypt [None req-f6574a67-a6de-4d30-8f41-c315ec03f4d2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80343) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 18:48:13.931998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG passlib.handlers.bcrypt [None req-f6574a67-a6de-4d30-8f41-c315ec03f4d2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80343) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 18:48:13.986549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-f6574a67-a6de-4d30-8f41-c315ec03f4d2 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:13.988284 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f6574a67-a6de-4d30-8f41-c315ec03f4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:14.112295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 2/3] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 14 18:48:14.313549 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f48995-33f9-48a3-be71-b87980023ccb None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:14.382015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: INFO dbcounter [None req-12f48995-33f9-48a3-be71-b87980023ccb None None] Registered counter for database keystone Jul 14 18:48:14.401229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG oslo_db.sqlalchemy.engines [None req-12f48995-33f9-48a3-be71-b87980023ccb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80342) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 18:48:14.407312 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writer thread running {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 18:48:14.523769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f48995-33f9-48a3-be71-b87980023ccb None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:14.528408 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d547d2-8ecd-40e4-8f96-40f942e732dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:14.529064 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d547d2-8ecd-40e4-8f96-40f942e732dd admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:14.529354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d547d2-8ecd-40e4-8f96-40f942e732dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:14.529631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d547d2-8ecd-40e4-8f96-40f942e732dd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:14.538135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d547d2-8ecd-40e4-8f96-40f942e732dd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:14.589514 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c1d547d2-8ecd-40e4-8f96-40f942e732dd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 18:48:14.589858 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d547d2-8ecd-40e4-8f96-40f942e732dd admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:14.599803 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 2/4] 10.222.0.69 () {58 vars in 1324 bytes} [Thu Jul 14 18:48:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 18:48:14.605835 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6574a67-a6de-4d30-8f41-c315ec03f4d2 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:14.620774 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6574a67-a6de-4d30-8f41-c315ec03f4d2 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:14.625109 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b10bd7-c53a-4aeb-98d6-0f6a9cc78e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:14.626235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b10bd7-c53a-4aeb-98d6-0f6a9cc78e64 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:14.626525 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b10bd7-c53a-4aeb-98d6-0f6a9cc78e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:14.626804 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b10bd7-c53a-4aeb-98d6-0f6a9cc78e64 admin admin] PATH_INFO: `/v3/projects/1ecc6a6cebab48d4b8cfff094ca5066b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:14.630869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b10bd7-c53a-4aeb-98d6-0f6a9cc78e64 admin admin] RBAC: Authorizing `identity:get_project(project_id=1ecc6a6cebab48d4b8cfff094ca5066b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:14.681743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b9b10bd7-c53a-4aeb-98d6-0f6a9cc78e64 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 18:48:14.682280 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b10bd7-c53a-4aeb-98d6-0f6a9cc78e64 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:14.686657 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 3/5] 10.222.0.69 () {56 vars in 1207 bytes} [Thu Jul 14 18:48:14 2022] GET /identity/v3/projects/1ecc6a6cebab48d4b8cfff094ca5066b => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:16.627586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b1186-e4ca-47c7-9736-f5069ed0e52c None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:16.628127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b1186-e4ca-47c7-9736-f5069ed0e52c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:16.628127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b1186-e4ca-47c7-9736-f5069ed0e52c None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:16.632880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 3/6] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:16.638389 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a0c0db-1916-4515-8df0-8e51cc529787 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:16.638782 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a0c0db-1916-4515-8df0-8e51cc529787 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:16.638959 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a0c0db-1916-4515-8df0-8e51cc529787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:16.657777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-26a0c0db-1916-4515-8df0-8e51cc529787 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:16.659328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-26a0c0db-1916-4515-8df0-8e51cc529787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:16.667453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 4/7] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 14 18:48:16.690412 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b1186-e4ca-47c7-9736-f5069ed0e52c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:16.702978 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b1186-e4ca-47c7-9736-f5069ed0e52c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:16.707202 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c11d9-f3c3-4a3a-8067-18224e1be878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:16.707840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c11d9-f3c3-4a3a-8067-18224e1be878 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:16.708130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c11d9-f3c3-4a3a-8067-18224e1be878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:16.708410 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c11d9-f3c3-4a3a-8067-18224e1be878 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:16.716215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c11d9-f3c3-4a3a-8067-18224e1be878 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:16.718117 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c11d9-f3c3-4a3a-8067-18224e1be878 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:16.726909 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 4/8] 10.222.0.69 () {58 vars in 1324 bytes} [Thu Jul 14 18:48:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 18:48:16.732153 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a0c0db-1916-4515-8df0-8e51cc529787 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:16.741634 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a0c0db-1916-4515-8df0-8e51cc529787 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:16.746054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0119ab0-2c88-4015-b314-48b9026cc238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:16.746914 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0119ab0-2c88-4015-b314-48b9026cc238 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:16.747235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0119ab0-2c88-4015-b314-48b9026cc238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:16.747515 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0119ab0-2c88-4015-b314-48b9026cc238 admin admin] PATH_INFO: `/v3/users/82583283d3254dca9292c9de336ad74a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:16.748912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0119ab0-2c88-4015-b314-48b9026cc238 admin admin] RBAC: Authorizing `identity:get_user(user_id=82583283d3254dca9292c9de336ad74a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:16.750744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0119ab0-2c88-4015-b314-48b9026cc238 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:16.752362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 5/9] 10.222.0.69 () {56 vars in 1198 bytes} [Thu Jul 14 18:48:16 2022] GET /identity/v3/users/82583283d3254dca9292c9de336ad74a => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:18.770046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88968751-ebdb-491d-9c3b-83e60d0e7173 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:18.770589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88968751-ebdb-491d-9c3b-83e60d0e7173 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:18.770683 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88968751-ebdb-491d-9c3b-83e60d0e7173 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:18.772696 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 5/10] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:18.778076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ad9c8-f19d-4b2d-be8d-ff0a205710d9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:18.778492 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ad9c8-f19d-4b2d-be8d-ff0a205710d9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:18.778655 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ad9c8-f19d-4b2d-be8d-ff0a205710d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:18.795677 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6e1ad9c8-f19d-4b2d-be8d-ff0a205710d9 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:18.796887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6e1ad9c8-f19d-4b2d-be8d-ff0a205710d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:18.839785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cbaf8-fbd8-430f-9783-08f487426114 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:18.840970 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cbaf8-fbd8-430f-9783-08f487426114 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:18.841261 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cbaf8-fbd8-430f-9783-08f487426114 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:18.842632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 6/11] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:48:18.847993 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93542427-cf16-4c9b-81e8-3a5c3c143da2 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:18.848288 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93542427-cf16-4c9b-81e8-3a5c3c143da2 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:18.848931 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93542427-cf16-4c9b-81e8-3a5c3c143da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:18.879138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 6/12] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:18.889204 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG passlib.handlers.bcrypt [None req-93542427-cf16-4c9b-81e8-3a5c3c143da2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80342) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 18:48:18.889624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG passlib.handlers.bcrypt [None req-93542427-cf16-4c9b-81e8-3a5c3c143da2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80342) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 18:48:18.902312 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ad9c8-f19d-4b2d-be8d-ff0a205710d9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:18.925999 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:18.936132 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-93542427-cf16-4c9b-81e8-3a5c3c143da2 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:18.937746 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-93542427-cf16-4c9b-81e8-3a5c3c143da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:19.004602 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ad9c8-f19d-4b2d-be8d-ff0a205710d9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:19.038904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f91e82-820b-4acc-b5cc-c02520566d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:19.039266 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 7/13] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:19.039670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f91e82-820b-4acc-b5cc-c02520566d7e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:19.039957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f91e82-820b-4acc-b5cc-c02520566d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:19.040237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f91e82-820b-4acc-b5cc-c02520566d7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:19.051193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f91e82-820b-4acc-b5cc-c02520566d7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:19.060745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93542427-cf16-4c9b-81e8-3a5c3c143da2 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:19.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-71f91e82-820b-4acc-b5cc-c02520566d7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:48:19.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:19.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:19.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:19.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:19.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:19.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:19.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:19.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:19.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:19.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:19.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:19.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:19.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:19.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:19.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:19.063769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:19.063769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:19.063769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:19.063769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:19.063769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:19.063769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:19.063769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:19.063769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:19.063769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:19.063769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:19.063769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:19.063769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:19.063769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:19.063769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:19.063769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:19.063769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:19.064609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:19.064609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:19.064609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:19.064609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:19.064609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:19.064609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:19.064609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:19.064609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:19.064609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:19.064609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:19.064609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:48:19.064609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:19.064609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 7/14] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:48:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:48:19.070103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f91e82-820b-4acc-b5cc-c02520566d7e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:19.150875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:19.162747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93542427-cf16-4c9b-81e8-3a5c3c143da2 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:19.165129 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:19.168549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f91e82-820b-4acc-b5cc-c02520566d7e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:19.200094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79d0bf-1219-42c7-b66c-d122c4920a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:19.201068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79d0bf-1219-42c7-b66c-d122c4920a43 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:19.201363 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79d0bf-1219-42c7-b66c-d122c4920a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:19.201699 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79d0bf-1219-42c7-b66c-d122c4920a43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:19.202972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02928389-ac79-449a-8951-77ae76f44d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:19.203663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02928389-ac79-449a-8951-77ae76f44d2d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:19.203952 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02928389-ac79-449a-8951-77ae76f44d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:19.204232 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02928389-ac79-449a-8951-77ae76f44d2d None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:19.205004 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02928389-ac79-449a-8951-77ae76f44d2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:19.205320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02928389-ac79-449a-8951-77ae76f44d2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:19.206732 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02928389-ac79-449a-8951-77ae76f44d2d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:19.223914 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 8/15] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:19.229756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02928389-ac79-449a-8951-77ae76f44d2d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:19.291641 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79d0bf-1219-42c7-b66c-d122c4920a43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:19.293080 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79d0bf-1219-42c7-b66c-d122c4920a43 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:19.335422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02928389-ac79-449a-8951-77ae76f44d2d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:19.370879 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa22cc0-b360-4289-a37a-0f8178b04683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:19.371892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa22cc0-b360-4289-a37a-0f8178b04683 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:19.372317 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa22cc0-b360-4289-a37a-0f8178b04683 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:19.372598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa22cc0-b360-4289-a37a-0f8178b04683 None admin] PATH_INFO: `/v3/users/82583283d3254dca9292c9de336ad74a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:19.373973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa22cc0-b360-4289-a37a-0f8178b04683 None admin] RBAC: Authorizing `identity:get_user(user_id=82583283d3254dca9292c9de336ad74a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:19.375740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa22cc0-b360-4289-a37a-0f8178b04683 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:19.377400 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 9/16] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:19 2022] GET /identity/v3/users/82583283d3254dca9292c9de336ad74a => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:19.383391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa22cc0-b360-4289-a37a-0f8178b04683 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:19.393213 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 8/17] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:48:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 18:48:19.398350 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:19.399133 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79d0bf-1219-42c7-b66c-d122c4920a43 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:19.402061 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:19.483464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa22cc0-b360-4289-a37a-0f8178b04683 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:19.498364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79d0bf-1219-42c7-b66c-d122c4920a43 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:19.517970 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab475511-0360-4c73-9f41-e7200138bb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:19.518860 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab475511-0360-4c73-9f41-e7200138bb70 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:19.519149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab475511-0360-4c73-9f41-e7200138bb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:19.519491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab475511-0360-4c73-9f41-e7200138bb70 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:19.520679 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab475511-0360-4c73-9f41-e7200138bb70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:19.522146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab475511-0360-4c73-9f41-e7200138bb70 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:19.523952 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 10/18] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:48:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:19.529647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab475511-0360-4c73-9f41-e7200138bb70 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:19.535765 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9db46a-5c6a-4cf7-9a15-69c69db83be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:19.536570 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9db46a-5c6a-4cf7-9a15-69c69db83be3 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:19.536859 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9db46a-5c6a-4cf7-9a15-69c69db83be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:19.537140 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9db46a-5c6a-4cf7-9a15-69c69db83be3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:19.556330 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9db46a-5c6a-4cf7-9a15-69c69db83be3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:19.557836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9db46a-5c6a-4cf7-9a15-69c69db83be3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:19.579527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-ff9db46a-5c6a-4cf7-9a15-69c69db83be3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:48:19.579527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:19.579527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:19.579527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:19.579527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:19.579527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:19.579527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:19.579527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:19.579527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:19.579527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:19.579527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:19.579527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:19.579527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:48:19.579527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:48:19.579527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:48:19.580589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:48:19.580589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:19.580589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:19.580589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:19.580589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:19.580589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:19.580589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:19.580589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:19.580589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:19.580589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:19.580589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:19.580589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:19.580589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:19.580589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:19.580589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:19.580589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:19.580589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:19.581942 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:48:19.581942 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:48:19.581942 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:48:19.581942 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:48:19.581942 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 9/19] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:19.586556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9db46a-5c6a-4cf7-9a15-69c69db83be3 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:19.620293 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:19.640059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab475511-0360-4c73-9f41-e7200138bb70 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:19.647888 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:19.673986 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27db22d-4002-4085-8e66-c48cd88f10ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:19.674653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27db22d-4002-4085-8e66-c48cd88f10ff None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:19.674944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27db22d-4002-4085-8e66-c48cd88f10ff None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:19.675251 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27db22d-4002-4085-8e66-c48cd88f10ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:19.676118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27db22d-4002-4085-8e66-c48cd88f10ff None admin] RBAC: Adding query filter params (role.id=27fc14631c494ca0b2dc7f708be016c2, scope.domain.id=default, user.id=82583283d3254dca9292c9de336ad74a) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:19.676434 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27db22d-4002-4085-8e66-c48cd88f10ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:19.678478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27db22d-4002-4085-8e66-c48cd88f10ff None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:19.685436 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 11/20] 10.222.0.69 () {56 vars in 1322 bytes} [Thu Jul 14 18:48:19 2022] GET /identity/v3/role_assignments?user.id=82583283d3254dca9292c9de336ad74a&scope.domain.id=default&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:19.691364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9db46a-5c6a-4cf7-9a15-69c69db83be3 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:19.729059 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c6fe2-0503-41d7-bb57-b635d883e6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:19.729762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c6fe2-0503-41d7-bb57-b635d883e6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:19.730055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c6fe2-0503-41d7-bb57-b635d883e6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:19.730341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c6fe2-0503-41d7-bb57-b635d883e6d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:19.731101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c6fe2-0503-41d7-bb57-b635d883e6d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:19.731447 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c6fe2-0503-41d7-bb57-b635d883e6d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:19.732910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c6fe2-0503-41d7-bb57-b635d883e6d9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:19.753156 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 10/21] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:21.867056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28625b9-e31a-4891-9c7d-1c58f7ae9dd0 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:21.867615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28625b9-e31a-4891-9c7d-1c58f7ae9dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:21.867707 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28625b9-e31a-4891-9c7d-1c58f7ae9dd0 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:21.869185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 12/22] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:21.874422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64e998-6afd-433a-98c5-9d9467c7996c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:21.875288 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64e998-6afd-433a-98c5-9d9467c7996c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:21.875767 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64e998-6afd-433a-98c5-9d9467c7996c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:21.900201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4f64e998-6afd-433a-98c5-9d9467c7996c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:21.901672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4f64e998-6afd-433a-98c5-9d9467c7996c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:21.999478 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 11/23] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:22.037528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28625b9-e31a-4891-9c7d-1c58f7ae9dd0 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:22.211999 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28625b9-e31a-4891-9c7d-1c58f7ae9dd0 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:22.266789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317dfc35-4284-4e26-8cfd-1ff6d310a0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:22.267969 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317dfc35-4284-4e26-8cfd-1ff6d310a0fd None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:22.268450 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317dfc35-4284-4e26-8cfd-1ff6d310a0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:22.268914 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317dfc35-4284-4e26-8cfd-1ff6d310a0fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:22.280734 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317dfc35-4284-4e26-8cfd-1ff6d310a0fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:22.288744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-317dfc35-4284-4e26-8cfd-1ff6d310a0fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:48:22.288744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:22.288744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:22.288744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:22.288744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:22.288744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:22.288744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:22.288744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:22.288744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:22.288744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:22.288744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:22.288744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:22.288744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:22.288744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:22.288744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:22.288744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:22.289552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:22.289552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:22.289552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:22.289552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:22.289552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:22.289552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:22.289552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:22.289552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:22.289552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:22.289552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:22.289552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:22.289552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:22.289552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:22.289552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:22.289552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:22.289552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:22.290308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:22.290308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:22.290308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:22.290308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:22.290308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:22.290308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:22.290308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:22.290308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:22.290308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:22.290308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:22.290308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:48:22.290308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:22.292960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 13/24] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:48:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:22.304276 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64e998-6afd-433a-98c5-9d9467c7996c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:22.331704 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613e064-c727-493a-b343-33a201220ffc None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:22.332351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613e064-c727-493a-b343-33a201220ffc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:22.332826 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613e064-c727-493a-b343-33a201220ffc None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:22.335759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 14/25] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:48:22.341885 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55615ffa-da9b-4eb4-aed8-0b4878a682a4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:22.342439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55615ffa-da9b-4eb4-aed8-0b4878a682a4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:22.342908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55615ffa-da9b-4eb4-aed8-0b4878a682a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:22.352692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:22.379357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-55615ffa-da9b-4eb4-aed8-0b4878a682a4 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:22.381135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-55615ffa-da9b-4eb4-aed8-0b4878a682a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:22.395263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:22.496749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64e998-6afd-433a-98c5-9d9467c7996c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:22.518637 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 15/26] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:22.520048 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca41faa-a3cb-4831-ab01-38abbf608f6d None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:22.520340 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca41faa-a3cb-4831-ab01-38abbf608f6d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:22.520621 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca41faa-a3cb-4831-ab01-38abbf608f6d None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:22.522219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 16/27] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:22.528260 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159540ce-6f91-4fda-8245-fd630d9b7142 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:22.528553 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159540ce-6f91-4fda-8245-fd630d9b7142 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:22.528832 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159540ce-6f91-4fda-8245-fd630d9b7142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:22.559274 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-159540ce-6f91-4fda-8245-fd630d9b7142 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:22.561144 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-159540ce-6f91-4fda-8245-fd630d9b7142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:22.576208 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0716bf3-f6c7-4241-9e04-bebbbdb463cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:22.576885 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0716bf3-f6c7-4241-9e04-bebbbdb463cf None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:22.577168 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0716bf3-f6c7-4241-9e04-bebbbdb463cf None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:22.577497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0716bf3-f6c7-4241-9e04-bebbbdb463cf None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:22.578258 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0716bf3-f6c7-4241-9e04-bebbbdb463cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:22.578655 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0716bf3-f6c7-4241-9e04-bebbbdb463cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:22.580125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0716bf3-f6c7-4241-9e04-bebbbdb463cf None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:22.601657 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 12/28] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:22.602837 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0716bf3-f6c7-4241-9e04-bebbbdb463cf None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:22.663197 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 17/29] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:22.663197 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159540ce-6f91-4fda-8245-fd630d9b7142 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:22.735902 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:22.744419 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0716bf3-f6c7-4241-9e04-bebbbdb463cf None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:22.754559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:22.774807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159540ce-6f91-4fda-8245-fd630d9b7142 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:22.782116 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078ae6b-b43c-4936-9d8a-4a961776a946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:22.782946 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078ae6b-b43c-4936-9d8a-4a961776a946 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:22.783323 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078ae6b-b43c-4936-9d8a-4a961776a946 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:22.783707 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078ae6b-b43c-4936-9d8a-4a961776a946 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:22.784645 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078ae6b-b43c-4936-9d8a-4a961776a946 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:22.786215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078ae6b-b43c-4936-9d8a-4a961776a946 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:22.813487 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd238c2-86b4-4814-b930-7b791d259104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:22.814369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd238c2-86b4-4814-b930-7b791d259104 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:22.815405 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd238c2-86b4-4814-b930-7b791d259104 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:22.815740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd238c2-86b4-4814-b930-7b791d259104 None admin] PATH_INFO: `/v3/users/82583283d3254dca9292c9de336ad74a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:22.816999 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd238c2-86b4-4814-b930-7b791d259104 None admin] RBAC: Authorizing `identity:get_user(user_id=82583283d3254dca9292c9de336ad74a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:22.818989 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd238c2-86b4-4814-b930-7b791d259104 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:22.820657 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 18/30] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:22 2022] GET /identity/v3/users/82583283d3254dca9292c9de336ad74a => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:22.822065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42767296-edfd-4292-8d01-714434e48dbb None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:22.834208 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42767296-edfd-4292-8d01-714434e48dbb None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:22.834208 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42767296-edfd-4292-8d01-714434e48dbb None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:22.834208 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 19/31] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:22.834208 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42767296-edfd-4292-8d01-714434e48dbb None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:22.941004 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42767296-edfd-4292-8d01-714434e48dbb None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:22.973095 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:22.978448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae8a96-bd98-46e6-a2be-33b6330832f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:22.979444 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae8a96-bd98-46e6-a2be-33b6330832f2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:22.979956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae8a96-bd98-46e6-a2be-33b6330832f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:22.980372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae8a96-bd98-46e6-a2be-33b6330832f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:22.989329 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 13/32] 10.222.0.69 () {60 vars in 1131 bytes} [Thu Jul 14 18:48:22 2022] POST /identity/v3/roles => generated 219 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:48:22.991071 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078ae6b-b43c-4936-9d8a-4a961776a946 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:22.992778 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae8a96-bd98-46e6-a2be-33b6330832f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:22.994642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae8a96-bd98-46e6-a2be-33b6330832f2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:23.017299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-3cae8a96-bd98-46e6-a2be-33b6330832f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:48:23.017299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:23.017299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:23.017299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:23.017299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:23.017299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:23.017299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:23.017299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:23.017299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:23.017299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:23.017299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:23.017299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:23.017299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:48:23.017299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:48:23.017299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:48:23.018161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:48:23.018161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:23.018161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:23.018161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:23.018161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:23.018161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:23.018161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:23.018161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:23.018161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:23.018161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:23.018161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:23.018161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:23.018161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:23.018161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:23.018161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:23.018161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:23.018161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:23.019005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:48:23.019005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:48:23.019005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:48:23.019005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:23.021783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 20/33] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:48:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:23.023427 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb360b7-5e07-4374-acb9-2dda69e36a21 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:23.023991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb360b7-5e07-4374-acb9-2dda69e36a21 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:23.024482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb360b7-5e07-4374-acb9-2dda69e36a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:23.055181 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-4fb360b7-5e07-4374-acb9-2dda69e36a21 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:23.056384 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4fb360b7-5e07-4374-acb9-2dda69e36a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:23.127414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078ae6b-b43c-4936-9d8a-4a961776a946 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:23.166660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55793755-7afc-4032-899f-f7001a4c2c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:23.167867 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55793755-7afc-4032-899f-f7001a4c2c50 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:23.168371 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55793755-7afc-4032-899f-f7001a4c2c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:23.168844 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55793755-7afc-4032-899f-f7001a4c2c50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:23.170722 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55793755-7afc-4032-899f-f7001a4c2c50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:23.172676 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55793755-7afc-4032-899f-f7001a4c2c50 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:23.174611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 14/34] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:48:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:23.175664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55793755-7afc-4032-899f-f7001a4c2c50 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:23.213936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 21/35] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:23.214916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb360b7-5e07-4374-acb9-2dda69e36a21 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:23.222237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:23.321486 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55793755-7afc-4032-899f-f7001a4c2c50 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:23.364049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40792b-8ff4-4d09-8cd9-e4140fbbe792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:23.365707 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40792b-8ff4-4d09-8cd9-e4140fbbe792 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:23.366220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40792b-8ff4-4d09-8cd9-e4140fbbe792 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:23.366592 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40792b-8ff4-4d09-8cd9-e4140fbbe792 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:23.367477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40792b-8ff4-4d09-8cd9-e4140fbbe792 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:23.367993 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40792b-8ff4-4d09-8cd9-e4140fbbe792 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:23.369588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40792b-8ff4-4d09-8cd9-e4140fbbe792 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:23.372590 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb360b7-5e07-4374-acb9-2dda69e36a21 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:23.383606 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 15/36] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:23.384554 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40792b-8ff4-4d09-8cd9-e4140fbbe792 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:23.398814 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:23.429678 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5d7e6-8d8d-4fc5-9b8e-3f3e44606b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:23.431980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5d7e6-8d8d-4fc5-9b8e-3f3e44606b58 None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:23.432635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5d7e6-8d8d-4fc5-9b8e-3f3e44606b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:23.433120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5d7e6-8d8d-4fc5-9b8e-3f3e44606b58 None admin] PATH_INFO: `/v3/domains/default/users/82583283d3254dca9292c9de336ad74a/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:23.435713 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5d7e6-8d8d-4fc5-9b8e-3f3e44606b58 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=82583283d3254dca9292c9de336ad74a, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:23.438004 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5d7e6-8d8d-4fc5-9b8e-3f3e44606b58 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:23.500413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:23.506935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40792b-8ff4-4d09-8cd9-e4140fbbe792 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:23.540111 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf1165-07ad-4a31-8dc3-144ca74db848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:23.540785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf1165-07ad-4a31-8dc3-144ca74db848 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:23.541072 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf1165-07ad-4a31-8dc3-144ca74db848 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:23.541351 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf1165-07ad-4a31-8dc3-144ca74db848 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:23.542103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf1165-07ad-4a31-8dc3-144ca74db848 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:23.543515 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf1165-07ad-4a31-8dc3-144ca74db848 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:23.616219 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 16/37] 10.222.0.69 () {60 vars in 1131 bytes} [Thu Jul 14 18:48:23 2022] POST /identity/v3/roles => generated 225 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:48:23.617243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf1165-07ad-4a31-8dc3-144ca74db848 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:23.649509 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 22/38] 10.222.0.69 () {58 vars in 1361 bytes} [Thu Jul 14 18:48:23 2022] PUT /identity/v3/domains/default/users/82583283d3254dca9292c9de336ad74a/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:48:23.651769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430dc67-847c-4c74-a5ee-3dfb9241850a None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:23.652065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430dc67-847c-4c74-a5ee-3dfb9241850a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:23.652350 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430dc67-847c-4c74-a5ee-3dfb9241850a None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:23.657144 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 23/39] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:23.663351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea00786-6171-428d-98f5-198907d83296 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:23.663652 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea00786-6171-428d-98f5-198907d83296 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:23.663934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea00786-6171-428d-98f5-198907d83296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:23.684189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0ea00786-6171-428d-98f5-198907d83296 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:23.686065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0ea00786-6171-428d-98f5-198907d83296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:23.760439 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf1165-07ad-4a31-8dc3-144ca74db848 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:23.839254 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bcf43e-a303-4efc-91d6-99208c8b7b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:23.840023 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcf43e-a303-4efc-91d6-99208c8b7b8a None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:23.840315 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcf43e-a303-4efc-91d6-99208c8b7b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:23.840598 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcf43e-a303-4efc-91d6-99208c8b7b8a None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:23.841396 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bcf43e-a303-4efc-91d6-99208c8b7b8a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:23.842904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bcf43e-a303-4efc-91d6-99208c8b7b8a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:23.894352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 24/40] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:23.917825 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea00786-6171-428d-98f5-198907d83296 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:23.985172 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 17/41] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 14 18:48:23 2022] POST /identity/v3/projects => generated 301 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:48:23.986625 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4e781-6ed1-43bd-a21b-11daa0fdf47b None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:23.986926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4e781-6ed1-43bd-a21b-11daa0fdf47b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:23.987333 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4e781-6ed1-43bd-a21b-11daa0fdf47b None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:23.991232 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 18/42] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:23.998890 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83dd20-16fa-43fc-8b2d-905a11724321 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:24.005044 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83dd20-16fa-43fc-8b2d-905a11724321 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:24.005581 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83dd20-16fa-43fc-8b2d-905a11724321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:24.031111 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3f83dd20-16fa-43fc-8b2d-905a11724321 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:24.034157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3f83dd20-16fa-43fc-8b2d-905a11724321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:24.035707 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea00786-6171-428d-98f5-198907d83296 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:24.039907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:24.049623 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:24.076365 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dfa32f-9086-4e71-ab98-379ad4859b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:24.077061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfa32f-9086-4e71-ab98-379ad4859b43 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:24.077794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfa32f-9086-4e71-ab98-379ad4859b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:24.078455 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfa32f-9086-4e71-ab98-379ad4859b43 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:24.080023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dfa32f-9086-4e71-ab98-379ad4859b43 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:24.082749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dfa32f-9086-4e71-ab98-379ad4859b43 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:24.124424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 19/43] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:24.149024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83dd20-16fa-43fc-8b2d-905a11724321 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:24.162858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 25/44] 10.222.0.69 () {60 vars in 1131 bytes} [Thu Jul 14 18:48:23 2022] POST /identity/v3/roles => generated 223 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:48:24.286070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83dd20-16fa-43fc-8b2d-905a11724321 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:24.325406 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2c745-0b2d-4853-ba74-efeb81e7649b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:24.326223 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2c745-0b2d-4853-ba74-efeb81e7649b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:24.326510 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2c745-0b2d-4853-ba74-efeb81e7649b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:24.326789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2c745-0b2d-4853-ba74-efeb81e7649b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:24.328017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2c745-0b2d-4853-ba74-efeb81e7649b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:24.329496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2c745-0b2d-4853-ba74-efeb81e7649b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:24.331138 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 20/45] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:48:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:24.337619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dfa32f-9086-4e71-ab98-379ad4859b43 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:24.442254 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dfa32f-9086-4e71-ab98-379ad4859b43 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:24.478138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f0b45-17ea-4b7f-8565-728b7e29e812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:24.478902 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f0b45-17ea-4b7f-8565-728b7e29e812 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:24.479218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f0b45-17ea-4b7f-8565-728b7e29e812 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:24.479507 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f0b45-17ea-4b7f-8565-728b7e29e812 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:24.480222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f0b45-17ea-4b7f-8565-728b7e29e812 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:24.481736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f0b45-17ea-4b7f-8565-728b7e29e812 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:24.554463 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 26/46] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 14 18:48:24 2022] POST /identity/v3/projects => generated 312 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:48:25.800601 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05595fa3-23fa-4511-ab8b-8478fb69253f None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:25.801116 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05595fa3-23fa-4511-ab8b-8478fb69253f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:25.801519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05595fa3-23fa-4511-ab8b-8478fb69253f None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:25.802921 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 21/47] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:25.808362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae541860-f75b-4396-b4b3-5923cc54bbb0 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:25.808780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae541860-f75b-4396-b4b3-5923cc54bbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:25.809005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae541860-f75b-4396-b4b3-5923cc54bbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:25.825943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ae541860-f75b-4396-b4b3-5923cc54bbb0 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:25.827121 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ae541860-f75b-4396-b4b3-5923cc54bbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:25.838218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:25.892251 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 27/48] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:25.914926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05595fa3-23fa-4511-ab8b-8478fb69253f None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:25.923722 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:26.120218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05595fa3-23fa-4511-ab8b-8478fb69253f None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:26.154287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8633f-508a-4042-9640-3b6c17d7f9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:26.155090 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8633f-508a-4042-9640-3b6c17d7f9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:26.155417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8633f-508a-4042-9640-3b6c17d7f9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:26.155702 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8633f-508a-4042-9640-3b6c17d7f9c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:26.166852 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8633f-508a-4042-9640-3b6c17d7f9c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:26.175781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b2f8633f-508a-4042-9640-3b6c17d7f9c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:48:26.175781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:26.175781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:26.175781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:26.175781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:26.175781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:26.175781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:26.175781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:26.175781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:26.175781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:26.175781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:26.175781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:26.175781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:26.175781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:26.175781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:26.175781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:26.176749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:26.176749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:26.176749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:26.176749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:26.176749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:26.176749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:26.176749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:26.176749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:26.176749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:26.176749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:26.176749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:26.176749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:26.176749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:26.176749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:26.176749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:26.176749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:26.178088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:26.178088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:26.178088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:26.178088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:26.178088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:26.178088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:26.178088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:26.178088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:26.178088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:26.178088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:26.178088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:48:26.178088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:48:26.178088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 22/49] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:48:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:26.183429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae541860-f75b-4396-b4b3-5923cc54bbb0 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:26.280108 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae541860-f75b-4396-b4b3-5923cc54bbb0 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:26.313302 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ecc1b-fcc2-413e-95f2-ac61e57e8488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:26.313972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ecc1b-fcc2-413e-95f2-ac61e57e8488 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:26.314264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ecc1b-fcc2-413e-95f2-ac61e57e8488 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:26.314547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ecc1b-fcc2-413e-95f2-ac61e57e8488 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:26.315328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ecc1b-fcc2-413e-95f2-ac61e57e8488 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:26.315642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ecc1b-fcc2-413e-95f2-ac61e57e8488 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:26.317044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ecc1b-fcc2-413e-95f2-ac61e57e8488 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:26.325944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 28/50] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:26.331511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8633f-508a-4042-9640-3b6c17d7f9c1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:26.396945 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:26.431465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8633f-508a-4042-9640-3b6c17d7f9c1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:26.465891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364dd79d-8d50-49d1-b2f5-b31ac8d10abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:26.466766 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364dd79d-8d50-49d1-b2f5-b31ac8d10abf None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:26.467058 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364dd79d-8d50-49d1-b2f5-b31ac8d10abf None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:26.467371 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364dd79d-8d50-49d1-b2f5-b31ac8d10abf None admin] PATH_INFO: `/v3/users/82583283d3254dca9292c9de336ad74a` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:26.468800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364dd79d-8d50-49d1-b2f5-b31ac8d10abf None admin] RBAC: Authorizing `identity:get_user(user_id=82583283d3254dca9292c9de336ad74a)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:26.470270 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364dd79d-8d50-49d1-b2f5-b31ac8d10abf None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:26.471976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 23/51] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:26 2022] GET /identity/v3/users/82583283d3254dca9292c9de336ad74a => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:26.478083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ecc1b-fcc2-413e-95f2-ac61e57e8488 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:26.489520 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:26.543344 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a84e0-3890-47b9-9bd5-8de94caa8aad None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:26.543644 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a84e0-3890-47b9-9bd5-8de94caa8aad None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:26.543935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a84e0-3890-47b9-9bd5-8de94caa8aad None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:26.545322 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 24/52] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:48:26.550198 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabbe12-269e-4dfe-97ec-8e715833b10a None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:26.550492 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabbe12-269e-4dfe-97ec-8e715833b10a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:26.550773 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabbe12-269e-4dfe-97ec-8e715833b10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:26.567262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-1cabbe12-269e-4dfe-97ec-8e715833b10a None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:26.568962 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1cabbe12-269e-4dfe-97ec-8e715833b10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:26.579118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ecc1b-fcc2-413e-95f2-ac61e57e8488 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:26.612261 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfb073-ecca-4855-88a1-46bb61382f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:26.613054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfb073-ecca-4855-88a1-46bb61382f27 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:26.613344 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfb073-ecca-4855-88a1-46bb61382f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:26.613623 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfb073-ecca-4855-88a1-46bb61382f27 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:26.614808 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfb073-ecca-4855-88a1-46bb61382f27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:26.616301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfb073-ecca-4855-88a1-46bb61382f27 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:26.617921 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 29/53] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:48:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:26.623531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfb073-ecca-4855-88a1-46bb61382f27 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:26.634939 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 25/54] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:48:26.657768 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabbe12-269e-4dfe-97ec-8e715833b10a None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:26.697797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:26.717540 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:26.719949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfb073-ecca-4855-88a1-46bb61382f27 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:26.755533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369415d-94a0-442a-84d8-327e8383db73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:26.755720 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabbe12-269e-4dfe-97ec-8e715833b10a None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:26.756209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369415d-94a0-442a-84d8-327e8383db73 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:26.756500 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369415d-94a0-442a-84d8-327e8383db73 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:26.756779 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369415d-94a0-442a-84d8-327e8383db73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:26.757616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369415d-94a0-442a-84d8-327e8383db73 None admin] RBAC: Adding query filter params (role.id=27fc14631c494ca0b2dc7f708be016c2, scope.domain.id=default, user.id=82583283d3254dca9292c9de336ad74a) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:26.757936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369415d-94a0-442a-84d8-327e8383db73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:26.759416 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369415d-94a0-442a-84d8-327e8383db73 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:26.765160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 30/55] 10.222.0.69 () {56 vars in 1322 bytes} [Thu Jul 14 18:48:26 2022] GET /identity/v3/role_assignments?user.id=82583283d3254dca9292c9de336ad74a&scope.domain.id=default&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:26.791644 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2ba9e-18c9-454d-824c-1711e6274da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:26.792484 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2ba9e-18c9-454d-824c-1711e6274da8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:26.792781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2ba9e-18c9-454d-824c-1711e6274da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:26.793064 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2ba9e-18c9-454d-824c-1711e6274da8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:26.794380 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2ba9e-18c9-454d-824c-1711e6274da8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:26.795944 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2ba9e-18c9-454d-824c-1711e6274da8 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:26.797648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 26/56] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:48:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:26.803718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369415d-94a0-442a-84d8-327e8383db73 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:26.908430 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369415d-94a0-442a-84d8-327e8383db73 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:26.945864 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25681ce-ec7c-40df-bc1d-2ea741c2e7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:26.946623 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25681ce-ec7c-40df-bc1d-2ea741c2e7b1 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:26.946909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25681ce-ec7c-40df-bc1d-2ea741c2e7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:26.947206 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25681ce-ec7c-40df-bc1d-2ea741c2e7b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:26.947926 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25681ce-ec7c-40df-bc1d-2ea741c2e7b1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:26.949367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25681ce-ec7c-40df-bc1d-2ea741c2e7b1 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:26.959696 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:27.011638 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 31/57] 10.222.0.69 () {60 vars in 1140 bytes} [Thu Jul 14 18:48:26 2022] POST /identity/v3/projects => generated 298 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:48:28.998168 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3dcbc9-4dab-497f-ba40-828c7cebb7bd None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:28.998699 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3dcbc9-4dab-497f-ba40-828c7cebb7bd None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:28.998789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3dcbc9-4dab-497f-ba40-828c7cebb7bd None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:29.000158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 27/58] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:29.005776 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b707b4b-eedd-4c50-85dd-5ef2e1fd1249 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:29.006130 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b707b4b-eedd-4c50-85dd-5ef2e1fd1249 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:29.006354 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b707b4b-eedd-4c50-85dd-5ef2e1fd1249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:29.023358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3b707b4b-eedd-4c50-85dd-5ef2e1fd1249 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:29.024571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3b707b4b-eedd-4c50-85dd-5ef2e1fd1249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:29.089192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 32/59] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:29.112214 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3dcbc9-4dab-497f-ba40-828c7cebb7bd None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:29.215187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3dcbc9-4dab-497f-ba40-828c7cebb7bd None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:29.243222 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:29.247821 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753296a-7339-4cc6-86e8-92c96b48924e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:29.248661 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753296a-7339-4cc6-86e8-92c96b48924e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:29.248900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753296a-7339-4cc6-86e8-92c96b48924e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:29.249180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753296a-7339-4cc6-86e8-92c96b48924e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:29.250354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753296a-7339-4cc6-86e8-92c96b48924e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:29.251840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753296a-7339-4cc6-86e8-92c96b48924e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:29.253458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 28/60] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:48:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:29.259724 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b707b4b-eedd-4c50-85dd-5ef2e1fd1249 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:29.354997 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b707b4b-eedd-4c50-85dd-5ef2e1fd1249 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:29.377062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:29.388832 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09775ac2-63c8-4368-9d01-f31c56c8f295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:29.389500 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09775ac2-63c8-4368-9d01-f31c56c8f295 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:29.389794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09775ac2-63c8-4368-9d01-f31c56c8f295 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:29.390078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09775ac2-63c8-4368-9d01-f31c56c8f295 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:29.390798 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09775ac2-63c8-4368-9d01-f31c56c8f295 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:29.392403 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09775ac2-63c8-4368-9d01-f31c56c8f295 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:29.517906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 33/61] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 14 18:48:29 2022] POST /identity/v3/users => generated 275 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:48:32.540971 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c468d-7f1e-42a1-b009-fa77f07ce946 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:32.541830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c468d-7f1e-42a1-b009-fa77f07ce946 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:32.542267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c468d-7f1e-42a1-b009-fa77f07ce946 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:32.543890 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 29/62] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:32.549184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30b7e5-c274-488f-8302-feda9e292d5d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:32.549985 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30b7e5-c274-488f-8302-feda9e292d5d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:32.550429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30b7e5-c274-488f-8302-feda9e292d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:32.571616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-1c30b7e5-c274-488f-8302-feda9e292d5d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:32.573084 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1c30b7e5-c274-488f-8302-feda9e292d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:32.668067 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb2901-ea14-4b3d-8d59-b862b31d14e1 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:32.668622 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb2901-ea14-4b3d-8d59-b862b31d14e1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:32.669054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb2901-ea14-4b3d-8d59-b862b31d14e1 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:32.670593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 30/63] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:32.673932 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 34/64] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:32.678510 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ced5c-b518-47a6-a7f4-cc68e7a6dc14 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:32.678510 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ced5c-b518-47a6-a7f4-cc68e7a6dc14 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:32.678510 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ced5c-b518-47a6-a7f4-cc68e7a6dc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:32.694513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c33ced5c-b518-47a6-a7f4-cc68e7a6dc14 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:32.696028 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c33ced5c-b518-47a6-a7f4-cc68e7a6dc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:32.713170 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30b7e5-c274-488f-8302-feda9e292d5d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:32.845403 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30b7e5-c274-488f-8302-feda9e292d5d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:32.848522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 31/65] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:32.883561 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:32.903861 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ced5c-b518-47a6-a7f4-cc68e7a6dc14 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:32.905164 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6135e2a-648c-42c7-8088-254c81ca2d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:32.911780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6135e2a-648c-42c7-8088-254c81ca2d6f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:32.911780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6135e2a-648c-42c7-8088-254c81ca2d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:32.911780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6135e2a-648c-42c7-8088-254c81ca2d6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:32.917861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6135e2a-648c-42c7-8088-254c81ca2d6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:32.925642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-b6135e2a-648c-42c7-8088-254c81ca2d6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:32.925642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:32.925642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:32.925642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:32.925642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:32.925642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:32.925642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:32.925642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:32.925642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:32.925642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:32.925642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:32.925642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:32.925642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:32.925642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:32.925642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:32.925642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:32.926491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:32.926491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:32.926491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:32.926491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:32.926491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:32.926491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:32.926491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:32.926491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:32.926491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:32.926491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:32.926491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:32.926491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:32.926491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:32.926491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:32.926491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:32.926491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:32.927965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:32.927965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:32.927965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:32.927965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:32.927965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:32.927965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:32.927965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:32.927965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:32.927965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:32.927965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:32.927965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:32.927965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:32.927965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 35/66] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:32.929132 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cc1ed-cce4-4ab8-b858-930d45a69b82 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:32.929427 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cc1ed-cce4-4ab8-b858-930d45a69b82 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:32.929709 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cc1ed-cce4-4ab8-b858-930d45a69b82 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:32.930987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 36/67] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:32.937024 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cc1ed-cce4-4ab8-b858-930d45a69b82 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:33.037708 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ced5c-b518-47a6-a7f4-cc68e7a6dc14 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:33.078533 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:33.082148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cc1ed-cce4-4ab8-b858-930d45a69b82 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:33.087791 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b6b63-3510-45f5-8a61-7d13db4f43ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:33.088605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b6b63-3510-45f5-8a61-7d13db4f43ca None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:33.088892 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b6b63-3510-45f5-8a61-7d13db4f43ca None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:33.089169 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b6b63-3510-45f5-8a61-7d13db4f43ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:33.103880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b6b63-3510-45f5-8a61-7d13db4f43ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:33.115107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-802b6b63-3510-45f5-8a61-7d13db4f43ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:48:33.115107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:33.115107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:33.115107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:33.115107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:33.115107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:33.115107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:33.115107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:33.115107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:33.115107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:33.115107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:33.115107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:33.115107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:33.115107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:33.115107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:33.115107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:33.115983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:33.115983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:33.115983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:33.115983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:33.115983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:33.115983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:33.115983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:33.115983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:33.115983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:33.115983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:33.115983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:33.115983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:33.115983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:33.115983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:33.115983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:33.115983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:33.116727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:33.116727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:33.116727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:33.116727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:33.116727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:33.116727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:33.116727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:33.116727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:33.116727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:33.116727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:33.116727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:48:33.116727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:48:33.119660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 32/68] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:48:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:33.121888 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfc0d2-e4ac-4302-afe2-769c5fdef2e5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:33.122355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfc0d2-e4ac-4302-afe2-769c5fdef2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:33.122806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfc0d2-e4ac-4302-afe2-769c5fdef2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:33.125317 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31f6c7-1618-4ee9-83f3-33694440a8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:33.125999 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31f6c7-1618-4ee9-83f3-33694440a8ab None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:33.126287 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31f6c7-1618-4ee9-83f3-33694440a8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:33.126598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31f6c7-1618-4ee9-83f3-33694440a8ab None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:33.127410 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31f6c7-1618-4ee9-83f3-33694440a8ab None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:33.127723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31f6c7-1618-4ee9-83f3-33694440a8ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:33.129315 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31f6c7-1618-4ee9-83f3-33694440a8ab None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:33.139485 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 37/69] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:48:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:33.140750 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acf58b-5a8c-43fa-a815-53f3caee5232 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:33.141043 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acf58b-5a8c-43fa-a815-53f3caee5232 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:33.141202 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-96cfc0d2-e4ac-4302-afe2-769c5fdef2e5 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:33.141320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acf58b-5a8c-43fa-a815-53f3caee5232 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:33.142342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-96cfc0d2-e4ac-4302-afe2-769c5fdef2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:33.142559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 38/70] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:33.147359 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39acf58b-5a8c-43fa-a815-53f3caee5232 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:33.180883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:33.223209 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 33/71] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:33.223209 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfc0d2-e4ac-4302-afe2-769c5fdef2e5 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:33.255123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39acf58b-5a8c-43fa-a815-53f3caee5232 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:33.294480 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15e7ec-ee2d-4475-9626-4bc79ca4d5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:33.295156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15e7ec-ee2d-4475-9626-4bc79ca4d5ed None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:33.295505 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15e7ec-ee2d-4475-9626-4bc79ca4d5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:33.295740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15e7ec-ee2d-4475-9626-4bc79ca4d5ed None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:33.296482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e15e7ec-ee2d-4475-9626-4bc79ca4d5ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:33.296794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e15e7ec-ee2d-4475-9626-4bc79ca4d5ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:33.298184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e15e7ec-ee2d-4475-9626-4bc79ca4d5ed None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:33.306738 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 39/72] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:33.308008 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8ea2-7c02-474c-9748-f2a9b0af463d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:33.308306 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8ea2-7c02-474c-9748-f2a9b0af463d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:33.308590 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8ea2-7c02-474c-9748-f2a9b0af463d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:33.325057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-968d8ea2-7c02-474c-9748-f2a9b0af463d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:33.326251 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-968d8ea2-7c02-474c-9748-f2a9b0af463d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:33.338576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfc0d2-e4ac-4302-afe2-769c5fdef2e5 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:33.359513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:33.372771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932847c-f74a-47af-b4d8-a1b567a44d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:33.373649 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932847c-f74a-47af-b4d8-a1b567a44d45 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:33.373944 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932847c-f74a-47af-b4d8-a1b567a44d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:33.376064 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932847c-f74a-47af-b4d8-a1b567a44d45 None admin] PATH_INFO: `/v3/users/e383b5a76d134501a90ca526cde31f54` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:33.389519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932847c-f74a-47af-b4d8-a1b567a44d45 None admin] RBAC: Authorizing `identity:get_user(user_id=e383b5a76d134501a90ca526cde31f54)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:33.390906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 40/73] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:33.403412 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932847c-f74a-47af-b4d8-a1b567a44d45 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:33.403512 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d8ea2-7c02-474c-9748-f2a9b0af463d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:33.403615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 34/74] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:33 2022] GET /identity/v3/users/e383b5a76d134501a90ca526cde31f54 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:33.403615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ff5a9-d5d3-4013-b20b-e9219f440b72 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:33.403615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ff5a9-d5d3-4013-b20b-e9219f440b72 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:33.403615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ff5a9-d5d3-4013-b20b-e9219f440b72 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:33.403615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 35/75] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:33.403615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ff5a9-d5d3-4013-b20b-e9219f440b72 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:33.431406 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:33.503420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ff5a9-d5d3-4013-b20b-e9219f440b72 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:33.504031 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d8ea2-7c02-474c-9748-f2a9b0af463d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:33.537788 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b19d53-86a9-45c5-b1ae-37aa93c68d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:33.538598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b19d53-86a9-45c5-b1ae-37aa93c68d3a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:33.538887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b19d53-86a9-45c5-b1ae-37aa93c68d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:33.539186 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b19d53-86a9-45c5-b1ae-37aa93c68d3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:33.540742 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74995354-b0a2-4e7f-aed1-8ef1fc2fa2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:33.541614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74995354-b0a2-4e7f-aed1-8ef1fc2fa2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:33.542148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74995354-b0a2-4e7f-aed1-8ef1fc2fa2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:33.542148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74995354-b0a2-4e7f-aed1-8ef1fc2fa2c8 None admin] PATH_INFO: `/v3/users/82583283d3254dca9292c9de336ad74a` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:33.543402 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74995354-b0a2-4e7f-aed1-8ef1fc2fa2c8 None admin] RBAC: Authorizing `identity:get_user(user_id=82583283d3254dca9292c9de336ad74a)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:33.544873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74995354-b0a2-4e7f-aed1-8ef1fc2fa2c8 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:33.546509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 36/76] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:33 2022] GET /identity/v3/users/82583283d3254dca9292c9de336ad74a => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:33.547306 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74995354-b0a2-4e7f-aed1-8ef1fc2fa2c8 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:33.550330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b19d53-86a9-45c5-b1ae-37aa93c68d3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:33.558449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-51b19d53-86a9-45c5-b1ae-37aa93c68d3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:48:33.558449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:33.558449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:33.558449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:33.558449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:33.558449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:33.558449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:33.558449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:33.558449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:33.558449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:33.558449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:33.558449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:33.558449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:33.558449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:33.558449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:33.558449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:33.559337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:33.559337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:33.559337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:33.559337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:33.559337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:33.559337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:33.559337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:33.559337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:33.559337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:33.559337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:33.559337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:33.559337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:33.559337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:33.559337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:33.559337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:33.559337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:33.560083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:33.560083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:33.560083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:33.560083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:33.560083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:33.560083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:33.560083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:33.560083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:33.560083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:33.560083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:33.560083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:48:33.560083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:33.560083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 41/77] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:48:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:33.561175 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e562b2a-542b-4a39-b14a-c591e2b96270 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:33.561464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e562b2a-542b-4a39-b14a-c591e2b96270 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:33.561747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e562b2a-542b-4a39-b14a-c591e2b96270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:33.579234 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-4e562b2a-542b-4a39-b14a-c591e2b96270 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:33.580368 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4e562b2a-542b-4a39-b14a-c591e2b96270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:33.598541 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:33.646175 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 42/78] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:33.647308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e562b2a-542b-4a39-b14a-c591e2b96270 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:33.653293 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74995354-b0a2-4e7f-aed1-8ef1fc2fa2c8 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:33.663089 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:33.688898 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26acfc2-e1ef-47c7-84a4-76027a3f973b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:33.689558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26acfc2-e1ef-47c7-84a4-76027a3f973b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:33.689854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26acfc2-e1ef-47c7-84a4-76027a3f973b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:33.690142 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26acfc2-e1ef-47c7-84a4-76027a3f973b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:33.746844 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e562b2a-542b-4a39-b14a-c591e2b96270 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:33.757342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26acfc2-e1ef-47c7-84a4-76027a3f973b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:33.758819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26acfc2-e1ef-47c7-84a4-76027a3f973b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:33.780992 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410b2fa-32ef-4f71-b6b9-e70d1b8fa642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:33.781828 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410b2fa-32ef-4f71-b6b9-e70d1b8fa642 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:33.782123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410b2fa-32ef-4f71-b6b9-e70d1b8fa642 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:33.782406 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410b2fa-32ef-4f71-b6b9-e70d1b8fa642 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:33.792523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410b2fa-32ef-4f71-b6b9-e70d1b8fa642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:33.800413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-2410b2fa-32ef-4f71-b6b9-e70d1b8fa642 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:33.800413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:33.800413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:33.800413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:33.800413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:33.800413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:33.800413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:33.800413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:33.800413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:33.800413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:33.800413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:33.800413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:33.800413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:33.800413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:33.800413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:33.800413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:33.801502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:33.801502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:33.801502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:33.801502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:33.801502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:33.801502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:33.801502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:33.801502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:33.801502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:33.801502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:33.801502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:33.801502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:33.801502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:33.801502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:33.801502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:33.801502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:33.802758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:33.802758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:33.802758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:33.802758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:33.802758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:33.802758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:33.802758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:33.802758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:33.802758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:33.802758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:33.802758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:33.802758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:33.802758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 43/79] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:33.803727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410b2fa-32ef-4f71-b6b9-e70d1b8fa642 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:33.804147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:33.859482 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 37/80] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:48:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:48:33.860319 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26acfc2-e1ef-47c7-84a4-76027a3f973b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:33.881936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:33.911472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410b2fa-32ef-4f71-b6b9-e70d1b8fa642 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:33.944665 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec8d3a-086e-44ff-b257-b62251b57b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:33.945322 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec8d3a-086e-44ff-b257-b62251b57b77 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:33.945615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec8d3a-086e-44ff-b257-b62251b57b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:33.945900 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec8d3a-086e-44ff-b257-b62251b57b77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:33.972195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26acfc2-e1ef-47c7-84a4-76027a3f973b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:34.006800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f5d56-c580-4d95-a152-e6a1c9d78d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:34.007508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f5d56-c580-4d95-a152-e6a1c9d78d09 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:34.007800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f5d56-c580-4d95-a152-e6a1c9d78d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:34.008076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f5d56-c580-4d95-a152-e6a1c9d78d09 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:34.008821 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f5d56-c580-4d95-a152-e6a1c9d78d09 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:34.009131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f5d56-c580-4d95-a152-e6a1c9d78d09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:34.010515 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f5d56-c580-4d95-a152-e6a1c9d78d09 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:34.012958 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec8d3a-086e-44ff-b257-b62251b57b77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:34.014442 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec8d3a-086e-44ff-b257-b62251b57b77 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:34.018483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 38/81] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:48:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:34.018483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f5d56-c580-4d95-a152-e6a1c9d78d09 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:34.029763 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:34.084714 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:34.113822 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 44/82] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:48:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:48:34.115371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec8d3a-086e-44ff-b257-b62251b57b77 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:34.117300 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f5d56-c580-4d95-a152-e6a1c9d78d09 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:34.150183 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99ec0c-d164-4c09-b174-5f7fbd713ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:34.151275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99ec0c-d164-4c09-b174-5f7fbd713ceb None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:34.151570 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99ec0c-d164-4c09-b174-5f7fbd713ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:34.151853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99ec0c-d164-4c09-b174-5f7fbd713ceb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:34.153068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99ec0c-d164-4c09-b174-5f7fbd713ceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:34.154532 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99ec0c-d164-4c09-b174-5f7fbd713ceb None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:34.156209 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 39/83] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:48:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:34.160848 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99ec0c-d164-4c09-b174-5f7fbd713ceb None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:34.214864 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec8d3a-086e-44ff-b257-b62251b57b77 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:34.233085 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:34.249103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce3b39-ca26-444b-88ef-9b49f6b21a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:34.249775 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce3b39-ca26-444b-88ef-9b49f6b21a29 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:34.250062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce3b39-ca26-444b-88ef-9b49f6b21a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:34.250348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce3b39-ca26-444b-88ef-9b49f6b21a29 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:34.251092 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcce3b39-ca26-444b-88ef-9b49f6b21a29 None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:34.251434 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcce3b39-ca26-444b-88ef-9b49f6b21a29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:34.252835 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcce3b39-ca26-444b-88ef-9b49f6b21a29 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:34.256491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99ec0c-d164-4c09-b174-5f7fbd713ceb None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:34.261338 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 45/84] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:48:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:34.262129 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce3b39-ca26-444b-88ef-9b49f6b21a29 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:34.295589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2916ec-74db-4912-ad02-44c71a9e20ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:34.296454 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2916ec-74db-4912-ad02-44c71a9e20ae None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:34.296753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2916ec-74db-4912-ad02-44c71a9e20ae None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:34.296883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:34.297089 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2916ec-74db-4912-ad02-44c71a9e20ae None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:34.298265 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2916ec-74db-4912-ad02-44c71a9e20ae None admin] RBAC: Authorizing `identity:get_project(project_id=b82ee754849c4779b408e856c429ab9b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:34.299759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2916ec-74db-4912-ad02-44c71a9e20ae None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:34.301508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 40/85] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:34 2022] GET /identity/v3/projects/b82ee754849c4779b408e856c429ab9b => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:34.302842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2916ec-74db-4912-ad02-44c71a9e20ae None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:34.364110 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce3b39-ca26-444b-88ef-9b49f6b21a29 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:34.398532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df95133-cbf5-4524-9451-79a7bf99d448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:34.399448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df95133-cbf5-4524-9451-79a7bf99d448 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:34.399746 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df95133-cbf5-4524-9451-79a7bf99d448 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:34.400028 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df95133-cbf5-4524-9451-79a7bf99d448 None admin] PATH_INFO: `/v3/users/e383b5a76d134501a90ca526cde31f54` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:34.401489 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df95133-cbf5-4524-9451-79a7bf99d448 None admin] RBAC: Authorizing `identity:get_user(user_id=e383b5a76d134501a90ca526cde31f54)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:34.402960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df95133-cbf5-4524-9451-79a7bf99d448 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:34.404647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 46/86] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:34 2022] GET /identity/v3/users/e383b5a76d134501a90ca526cde31f54 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:34.404742 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2916ec-74db-4912-ad02-44c71a9e20ae None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:34.405398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df95133-cbf5-4524-9451-79a7bf99d448 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:34.438979 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf7234-dedc-47dc-a88c-dad2eaa13b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:34.439892 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf7234-dedc-47dc-a88c-dad2eaa13b42 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:34.440184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf7234-dedc-47dc-a88c-dad2eaa13b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:34.440473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf7234-dedc-47dc-a88c-dad2eaa13b42 None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:34.441668 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf7234-dedc-47dc-a88c-dad2eaa13b42 None admin] RBAC: Authorizing `identity:get_project(project_id=b82ee754849c4779b408e856c429ab9b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:34.443147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf7234-dedc-47dc-a88c-dad2eaa13b42 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:34.444882 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 41/87] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:34 2022] GET /identity/v3/projects/b82ee754849c4779b408e856c429ab9b => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:34.445643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf7234-dedc-47dc-a88c-dad2eaa13b42 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:34.461546 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:34.509067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df95133-cbf5-4524-9451-79a7bf99d448 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:34.510510 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:34.543580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abfc19-e70c-413e-be56-60d0e6afb908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:34.544256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abfc19-e70c-413e-be56-60d0e6afb908 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:34.544544 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abfc19-e70c-413e-be56-60d0e6afb908 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:34.544827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abfc19-e70c-413e-be56-60d0e6afb908 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:34.545530 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82abfc19-e70c-413e-be56-60d0e6afb908 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:34.546651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf7234-dedc-47dc-a88c-dad2eaa13b42 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:34.547072 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82abfc19-e70c-413e-be56-60d0e6afb908 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:34.579583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41302eef-4e82-486f-a4b4-f6a59c270c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:34.580484 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41302eef-4e82-486f-a4b4-f6a59c270c3a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:34.580779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41302eef-4e82-486f-a4b4-f6a59c270c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:34.581066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41302eef-4e82-486f-a4b4-f6a59c270c3a None admin] PATH_INFO: `/v3/users/e383b5a76d134501a90ca526cde31f54` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:34.582282 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41302eef-4e82-486f-a4b4-f6a59c270c3a None admin] RBAC: Authorizing `identity:get_user(user_id=e383b5a76d134501a90ca526cde31f54)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:34.584348 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41302eef-4e82-486f-a4b4-f6a59c270c3a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:34.585993 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 42/88] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:34 2022] GET /identity/v3/users/e383b5a76d134501a90ca526cde31f54 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:34.586764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41302eef-4e82-486f-a4b4-f6a59c270c3a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:34.644422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 47/89] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 14 18:48:34 2022] POST /identity/v3/users => generated 289 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:48:34.645275 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abfc19-e70c-413e-be56-60d0e6afb908 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:34.674298 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:34.699613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41302eef-4e82-486f-a4b4-f6a59c270c3a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:34.745415 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ccf9b-cdae-4bff-aa7c-cbd37ea6751f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:34.746094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ccf9b-cdae-4bff-aa7c-cbd37ea6751f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:34.746386 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ccf9b-cdae-4bff-aa7c-cbd37ea6751f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:34.746846 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ccf9b-cdae-4bff-aa7c-cbd37ea6751f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:34.747803 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ccf9b-cdae-4bff-aa7c-cbd37ea6751f None admin] RBAC: Adding query filter params (role.id=ffed4493824a4040977d2cd1d057e07b, scope.project.id=b82ee754849c4779b408e856c429ab9b, user.id=e383b5a76d134501a90ca526cde31f54) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:34.748253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ccf9b-cdae-4bff-aa7c-cbd37ea6751f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:34.749917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ccf9b-cdae-4bff-aa7c-cbd37ea6751f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:34.760315 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 43/90] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:48:34 2022] GET /identity/v3/role_assignments?user.id=e383b5a76d134501a90ca526cde31f54&scope.project.id=b82ee754849c4779b408e856c429ab9b&role.id=ffed4493824a4040977d2cd1d057e07b => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:34.761132 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ccf9b-cdae-4bff-aa7c-cbd37ea6751f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:34.778042 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abfc19-e70c-413e-be56-60d0e6afb908 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:34.830867 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855620f-3010-4968-8577-cbda3ced20af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:34.831751 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855620f-3010-4968-8577-cbda3ced20af None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:34.832046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855620f-3010-4968-8577-cbda3ced20af None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:34.832328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855620f-3010-4968-8577-cbda3ced20af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:34.870791 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:34.913864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ccf9b-cdae-4bff-aa7c-cbd37ea6751f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:34.930147 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855620f-3010-4968-8577-cbda3ced20af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:34.932063 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855620f-3010-4968-8577-cbda3ced20af None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:34.978108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda71b1-8388-4caa-9222-8114e6f40eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:34.978788 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda71b1-8388-4caa-9222-8114e6f40eba None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:34.979080 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda71b1-8388-4caa-9222-8114e6f40eba None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:34.979391 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda71b1-8388-4caa-9222-8114e6f40eba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:34.980236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda71b1-8388-4caa-9222-8114e6f40eba None admin] RBAC: Adding query filter params (role.id=27fc14631c494ca0b2dc7f708be016c2, scope.project.id=b82ee754849c4779b408e856c429ab9b, user.id=82583283d3254dca9292c9de336ad74a) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:34.980556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda71b1-8388-4caa-9222-8114e6f40eba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:34.982033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda71b1-8388-4caa-9222-8114e6f40eba None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:34.989043 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 44/91] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:48:34 2022] GET /identity/v3/role_assignments?user.id=82583283d3254dca9292c9de336ad74a&scope.project.id=b82ee754849c4779b408e856c429ab9b&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:34.989736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda71b1-8388-4caa-9222-8114e6f40eba None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:35.017944 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:35.092221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 48/92] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:48:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:48:35.099583 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855620f-3010-4968-8577-cbda3ced20af None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:35.132750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda71b1-8388-4caa-9222-8114e6f40eba None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:35.175055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:35.187093 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45a7e4-bdf0-4ef3-a9e2-0e52615ad7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:35.188132 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45a7e4-bdf0-4ef3-a9e2-0e52615ad7bd None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:35.191112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45a7e4-bdf0-4ef3-a9e2-0e52615ad7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:35.191112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45a7e4-bdf0-4ef3-a9e2-0e52615ad7bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:35.224856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855620f-3010-4968-8577-cbda3ced20af None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:35.266301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9aa4bb-1983-4ac1-a4ba-01273ed3fff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:35.267357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9aa4bb-1983-4ac1-a4ba-01273ed3fff9 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:35.267723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9aa4bb-1983-4ac1-a4ba-01273ed3fff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:35.268128 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9aa4bb-1983-4ac1-a4ba-01273ed3fff9 None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:35.269501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9aa4bb-1983-4ac1-a4ba-01273ed3fff9 None admin] RBAC: Authorizing `identity:get_project(project_id=b82ee754849c4779b408e856c429ab9b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:35.271320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9aa4bb-1983-4ac1-a4ba-01273ed3fff9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:35.273448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 49/93] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:35 2022] GET /identity/v3/projects/b82ee754849c4779b408e856c429ab9b => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:35.279323 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9aa4bb-1983-4ac1-a4ba-01273ed3fff9 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:35.287426 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45a7e4-bdf0-4ef3-a9e2-0e52615ad7bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:35.288408 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45a7e4-bdf0-4ef3-a9e2-0e52615ad7bd None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:35.301523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:35.398983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9aa4bb-1983-4ac1-a4ba-01273ed3fff9 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:35.410625 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 45/94] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:48:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:48:35.419190 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45a7e4-bdf0-4ef3-a9e2-0e52615ad7bd None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:35.447902 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa5eff-14f0-429b-bacd-e1d13f2c89d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:35.449139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa5eff-14f0-429b-bacd-e1d13f2c89d5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:35.449573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa5eff-14f0-429b-bacd-e1d13f2c89d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:35.449888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa5eff-14f0-429b-bacd-e1d13f2c89d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:35.450871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa5eff-14f0-429b-bacd-e1d13f2c89d5 None admin] RBAC: Adding query filter params (role.id=394e165201dc41c8b3255de480821b1f, scope.project.id=b82ee754849c4779b408e856c429ab9b, user.id=e383b5a76d134501a90ca526cde31f54) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:35.451237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa5eff-14f0-429b-bacd-e1d13f2c89d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:35.453368 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa5eff-14f0-429b-bacd-e1d13f2c89d5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:35.459906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 50/95] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:48:35 2022] GET /identity/v3/role_assignments?user.id=e383b5a76d134501a90ca526cde31f54&scope.project.id=b82ee754849c4779b408e856c429ab9b&role.id=394e165201dc41c8b3255de480821b1f => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:35.460558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:35.615526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45a7e4-bdf0-4ef3-a9e2-0e52615ad7bd None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:35.637612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:35.669963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18d3d4-ec3f-46a8-b2e0-19125b775e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:35.671543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18d3d4-ec3f-46a8-b2e0-19125b775e0e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:35.671824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18d3d4-ec3f-46a8-b2e0-19125b775e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:35.672105 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18d3d4-ec3f-46a8-b2e0-19125b775e0e None admin] PATH_INFO: `/v3/projects/0c022d6570d94db6874a209a32bedefe` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:35.673633 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd18d3d4-ec3f-46a8-b2e0-19125b775e0e None admin] RBAC: Authorizing `identity:get_project(project_id=0c022d6570d94db6874a209a32bedefe)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:35.675096 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd18d3d4-ec3f-46a8-b2e0-19125b775e0e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:35.677509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 46/96] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:35 2022] GET /identity/v3/projects/0c022d6570d94db6874a209a32bedefe => generated 312 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:35.683438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa5eff-14f0-429b-bacd-e1d13f2c89d5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:35.784495 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa5eff-14f0-429b-bacd-e1d13f2c89d5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:35.823185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690580ff-4a0f-4b4d-a58e-ecc21ac53456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:35.823869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690580ff-4a0f-4b4d-a58e-ecc21ac53456 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:35.824163 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690580ff-4a0f-4b4d-a58e-ecc21ac53456 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:35.824444 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690580ff-4a0f-4b4d-a58e-ecc21ac53456 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:35.825280 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690580ff-4a0f-4b4d-a58e-ecc21ac53456 None admin] RBAC: Adding query filter params (role.id=ffed4493824a4040977d2cd1d057e07b, scope.project.id=0c022d6570d94db6874a209a32bedefe, user.id=e383b5a76d134501a90ca526cde31f54) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:35.825596 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690580ff-4a0f-4b4d-a58e-ecc21ac53456 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:35.827048 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690580ff-4a0f-4b4d-a58e-ecc21ac53456 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:35.835418 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 51/97] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:48:35 2022] GET /identity/v3/role_assignments?user.id=e383b5a76d134501a90ca526cde31f54&scope.project.id=0c022d6570d94db6874a209a32bedefe&role.id=ffed4493824a4040977d2cd1d057e07b => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:37.642874 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288775e-dd65-46d5-b05c-f970f18c66b9 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:37.643411 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288775e-dd65-46d5-b05c-f970f18c66b9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:37.643485 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288775e-dd65-46d5-b05c-f970f18c66b9 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:37.644984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 47/98] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:37.652823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90c9c4-1e25-496c-9c75-26f8b74b43da None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:37.652823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90c9c4-1e25-496c-9c75-26f8b74b43da None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:37.652823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90c9c4-1e25-496c-9c75-26f8b74b43da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:37.669318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-4b90c9c4-1e25-496c-9c75-26f8b74b43da None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:37.670828 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4b90c9c4-1e25-496c-9c75-26f8b74b43da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:37.723326 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:37.742502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 52/99] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:37.766422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288775e-dd65-46d5-b05c-f970f18c66b9 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:37.890238 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288775e-dd65-46d5-b05c-f970f18c66b9 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:37.927821 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9f0fe-f155-4037-acfb-dd7eae9876ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:37.929465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9f0fe-f155-4037-acfb-dd7eae9876ce None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:37.929980 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9f0fe-f155-4037-acfb-dd7eae9876ce None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:37.930485 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9f0fe-f155-4037-acfb-dd7eae9876ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:37.931924 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9f0fe-f155-4037-acfb-dd7eae9876ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:37.933598 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9f0fe-f155-4037-acfb-dd7eae9876ce None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:37.935477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 48/100] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:48:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:37.941208 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90c9c4-1e25-496c-9c75-26f8b74b43da None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:38.030344 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad40ada-2016-4203-8217-aafc8cff5ae6 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:38.030958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad40ada-2016-4203-8217-aafc8cff5ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:38.031481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad40ada-2016-4203-8217-aafc8cff5ae6 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:38.033086 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 49/101] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:38.039339 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd231a-a05c-415b-97e5-6dd53c45c874 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:38.039894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd231a-a05c-415b-97e5-6dd53c45c874 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:38.040370 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd231a-a05c-415b-97e5-6dd53c45c874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:38.047026 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90c9c4-1e25-496c-9c75-26f8b74b43da None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:38.058470 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-76fd231a-a05c-415b-97e5-6dd53c45c874 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:38.060111 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-76fd231a-a05c-415b-97e5-6dd53c45c874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:38.089392 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85b7cb-4b69-4408-84dd-eb249e813642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:38.090362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85b7cb-4b69-4408-84dd-eb249e813642 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:38.090661 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85b7cb-4b69-4408-84dd-eb249e813642 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:38.090955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85b7cb-4b69-4408-84dd-eb249e813642 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:38.091723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85b7cb-4b69-4408-84dd-eb249e813642 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:38.093988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85b7cb-4b69-4408-84dd-eb249e813642 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:38.113094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:38.136121 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 50/102] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:48:38.159681 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd231a-a05c-415b-97e5-6dd53c45c874 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:38.203552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 53/103] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 14 18:48:37 2022] POST /identity/v3/projects => generated 302 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:48:38.204866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06affaad-f3e4-413e-abbe-f6a62e575358 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:38.205174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06affaad-f3e4-413e-abbe-f6a62e575358 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:38.205461 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06affaad-f3e4-413e-abbe-f6a62e575358 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:38.206808 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 54/104] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:38.221433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258a75f-c80a-4380-8a8c-0808b3027a28 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:38.221754 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258a75f-c80a-4380-8a8c-0808b3027a28 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:38.222007 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258a75f-c80a-4380-8a8c-0808b3027a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:38.248717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-a258a75f-c80a-4380-8a8c-0808b3027a28 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:38.249935 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a258a75f-c80a-4380-8a8c-0808b3027a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:38.307029 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:38.351879 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd231a-a05c-415b-97e5-6dd53c45c874 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:38.376955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 55/105] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 14 18:48:38.404476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e318fa4-6826-4643-9490-4912cc90d385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:38.405569 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258a75f-c80a-4380-8a8c-0808b3027a28 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:38.405722 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e318fa4-6826-4643-9490-4912cc90d385 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:38.405967 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e318fa4-6826-4643-9490-4912cc90d385 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:38.406250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e318fa4-6826-4643-9490-4912cc90d385 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:38.420918 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e318fa4-6826-4643-9490-4912cc90d385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:38.429703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-5e318fa4-6826-4643-9490-4912cc90d385 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 18:48:38.429703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:38.429703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:38.429703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:38.429703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:38.429703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:38.429703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:38.429703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:38.429703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:38.429703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:38.429703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:38.429703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:38.429703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:38.429703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:38.429703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:38.429703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:38.430562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:38.430562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:38.430562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:38.430562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:38.430562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:38.430562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:38.430562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:38.430562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:38.430562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:38.430562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:38.430562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:38.430562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:38.430562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:38.430562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:38.430562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:38.430562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:38.431747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:38.431747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:38.431747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:38.431747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:38.431747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:38.431747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:38.431747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:38.431747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:38.431747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:38.431747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:38.431747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 18:48:38.431747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:48:38.431747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 51/106] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:48:38.436753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e318fa4-6826-4643-9490-4912cc90d385 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:38.488667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:38.549440 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258a75f-c80a-4380-8a8c-0808b3027a28 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:38.563442 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e318fa4-6826-4643-9490-4912cc90d385 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:38.588810 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b9eaf-76cc-4781-98f4-25441b458563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:38.589734 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b9eaf-76cc-4781-98f4-25441b458563 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:38.590031 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b9eaf-76cc-4781-98f4-25441b458563 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:38.590309 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b9eaf-76cc-4781-98f4-25441b458563 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:38.600638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c2730-9fad-40be-b5fa-92eb178522a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:38.601313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c2730-9fad-40be-b5fa-92eb178522a8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:38.601656 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c2730-9fad-40be-b5fa-92eb178522a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:38.601937 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c2730-9fad-40be-b5fa-92eb178522a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:38.602740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c2730-9fad-40be-b5fa-92eb178522a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:38.602931 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b9eaf-76cc-4781-98f4-25441b458563 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:38.603103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c2730-9fad-40be-b5fa-92eb178522a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:38.604746 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c2730-9fad-40be-b5fa-92eb178522a8 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:38.611718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-fe0b9eaf-76cc-4781-98f4-25441b458563 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:48:38.611718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:38.611718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:38.611718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:38.611718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:38.611718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:38.611718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:38.611718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:38.611718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:38.611718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:38.611718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:38.611718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:38.611718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:38.611718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:38.611718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:38.611718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:38.612566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:38.612566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:38.612566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:38.612566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:38.612566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:38.612566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:38.612566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:38.612566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:38.612566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:38.612566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:38.612566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:38.612566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:38.612566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:38.612566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:38.612566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:38.612566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:38.614224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:38.614224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:38.614224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:38.614224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:38.614224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:38.614224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:38.614224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:38.614224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:38.614224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:38.614224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:38.614224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:48:38.614224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:38.614224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 56/107] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:48:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:48:38.614841 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 52/108] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:48:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:38.614906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebad584-db8f-401d-ad2c-3b626bab0f0f None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:38.614906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebad584-db8f-401d-ad2c-3b626bab0f0f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:38.614906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebad584-db8f-401d-ad2c-3b626bab0f0f None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:38.620736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 57/109] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:38.620966 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c2730-9fad-40be-b5fa-92eb178522a8 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:38.621939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed13ca-b862-45fc-b112-299f96e51be0 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:38.622232 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed13ca-b862-45fc-b112-299f96e51be0 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:38.622515 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed13ca-b862-45fc-b112-299f96e51be0 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:38.623785 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 58/110] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:38.624482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed13ca-b862-45fc-b112-299f96e51be0 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:38.655742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:38.727895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:38.734534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c2730-9fad-40be-b5fa-92eb178522a8 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:38.741343 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed13ca-b862-45fc-b112-299f96e51be0 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:38.773151 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823f04b-b2d8-43ff-b8b9-631ab76ff2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:38.774135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823f04b-b2d8-43ff-b8b9-631ab76ff2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:38.774430 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823f04b-b2d8-43ff-b8b9-631ab76ff2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:38.774707 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823f04b-b2d8-43ff-b8b9-631ab76ff2b2 None admin] PATH_INFO: `/v3/users/a2489171853b43e8bc7ff724ab503ed9` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:38.782416 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1d7c3-aec1-4ab3-8d72-c21cbfb000f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:38.783328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1d7c3-aec1-4ab3-8d72-c21cbfb000f4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:38.783693 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1d7c3-aec1-4ab3-8d72-c21cbfb000f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:38.784065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1d7c3-aec1-4ab3-8d72-c21cbfb000f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:38.784895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1d7c3-aec1-4ab3-8d72-c21cbfb000f4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:38.785277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1d7c3-aec1-4ab3-8d72-c21cbfb000f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:38.787504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1d7c3-aec1-4ab3-8d72-c21cbfb000f4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:38.789542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823f04b-b2d8-43ff-b8b9-631ab76ff2b2 None admin] RBAC: Authorizing `identity:get_user(user_id=a2489171853b43e8bc7ff724ab503ed9)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:38.791190 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823f04b-b2d8-43ff-b8b9-631ab76ff2b2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:38.793358 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 53/111] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:38 2022] GET /identity/v3/users/a2489171853b43e8bc7ff724ab503ed9 => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:38.794604 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004690c-2fe5-4b74-8661-e64a98f79553 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:38.794896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004690c-2fe5-4b74-8661-e64a98f79553 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:38.795189 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004690c-2fe5-4b74-8661-e64a98f79553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:38.801499 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 59/112] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:48:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:38.802920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06adfc-dcc0-44bb-aa98-40807b3a8505 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:38.803259 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06adfc-dcc0-44bb-aa98-40807b3a8505 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:38.805381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06adfc-dcc0-44bb-aa98-40807b3a8505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:38.813256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3004690c-2fe5-4b74-8661-e64a98f79553 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:38.814467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3004690c-2fe5-4b74-8661-e64a98f79553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:38.822260 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7d06adfc-dcc0-44bb-aa98-40807b3a8505 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:38.823619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7d06adfc-dcc0-44bb-aa98-40807b3a8505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:38.886988 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 54/113] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:38.887694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004690c-2fe5-4b74-8661-e64a98f79553 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:38.907964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 60/114] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:38.908728 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06adfc-dcc0-44bb-aa98-40807b3a8505 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:38.935927 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:38.981542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:39.003343 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004690c-2fe5-4b74-8661-e64a98f79553 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:39.025264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06adfc-dcc0-44bb-aa98-40807b3a8505 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:39.041761 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b921284-df61-4104-b014-260abb985aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:39.042422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b921284-df61-4104-b014-260abb985aad None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:39.042716 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b921284-df61-4104-b014-260abb985aad None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:39.042995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b921284-df61-4104-b014-260abb985aad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:39.062554 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acd47b-b5d2-41ec-b56e-c508db6bc1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:39.063671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acd47b-b5d2-41ec-b56e-c508db6bc1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:39.064102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acd47b-b5d2-41ec-b56e-c508db6bc1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:39.064591 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acd47b-b5d2-41ec-b56e-c508db6bc1c4 None admin] PATH_INFO: `/v3/users/e383b5a76d134501a90ca526cde31f54` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:39.065793 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acd47b-b5d2-41ec-b56e-c508db6bc1c4 None admin] RBAC: Authorizing `identity:get_user(user_id=e383b5a76d134501a90ca526cde31f54)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:39.067415 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acd47b-b5d2-41ec-b56e-c508db6bc1c4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:39.074699 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 61/115] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:38 2022] GET /identity/v3/users/e383b5a76d134501a90ca526cde31f54 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:39.074699 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce13d8-7acd-45b5-8844-7f0d45decbaf None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:39.074699 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce13d8-7acd-45b5-8844-7f0d45decbaf None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:39.074699 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce13d8-7acd-45b5-8844-7f0d45decbaf None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:39.074699 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 62/116] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:39.074699 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce13d8-7acd-45b5-8844-7f0d45decbaf None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:39.122863 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b921284-df61-4104-b014-260abb985aad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:39.124423 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b921284-df61-4104-b014-260abb985aad None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:39.171509 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:39.187749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce13d8-7acd-45b5-8844-7f0d45decbaf None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:39.211920 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:39.225522 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458a55b-023e-42af-ba4d-287c6911b411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:39.226608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458a55b-023e-42af-ba4d-287c6911b411 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:39.226901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458a55b-023e-42af-ba4d-287c6911b411 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:39.227208 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458a55b-023e-42af-ba4d-287c6911b411 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:39.229249 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 55/117] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:48:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:48:39.229936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b921284-df61-4104-b014-260abb985aad None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:39.243438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458a55b-023e-42af-ba4d-287c6911b411 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:39.251420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-f458a55b-023e-42af-ba4d-287c6911b411 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:39.251420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:39.251420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:39.251420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:39.251420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:39.251420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:39.251420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:39.251420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:39.251420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:39.251420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:39.251420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:39.251420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:39.251420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:39.251420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:39.251420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:39.251420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:39.252267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:39.252267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:39.252267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:39.252267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:39.252267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:39.252267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:39.252267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:39.252267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:39.252267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:39.252267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:39.252267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:39.252267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:39.252267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:39.252267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:39.252267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:39.252267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:39.253016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:39.253016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:39.253016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:39.253016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:39.253016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:39.253016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:39.253016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:39.253016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:39.253016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:39.253016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:39.253016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:39.253016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:39.253016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 63/118] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:39.253688 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458a55b-023e-42af-ba4d-287c6911b411 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:39.344526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b921284-df61-4104-b014-260abb985aad None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:39.361098 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458a55b-023e-42af-ba4d-287c6911b411 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:39.379064 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5b02f-1bf5-446b-b77a-8c1657104dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:39.380594 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5b02f-1bf5-446b-b77a-8c1657104dbc None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:39.381027 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5b02f-1bf5-446b-b77a-8c1657104dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:39.381444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5b02f-1bf5-446b-b77a-8c1657104dbc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:39.392155 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb5b02f-1bf5-446b-b77a-8c1657104dbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:39.400660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-ccb5b02f-1bf5-446b-b77a-8c1657104dbc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:39.400660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:39.400660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:39.400660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:39.400660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:39.400660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:39.400660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:39.400660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:39.400660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:39.400660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:39.400660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:39.400660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:39.400660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:39.400660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:39.400660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:39.400660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:39.401926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:39.401926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:39.401926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:39.401926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:39.401926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:39.401926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:39.401926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:39.401926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:39.401926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:39.401926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:39.401926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:39.401926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:39.401926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:39.401926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:39.401926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:39.401926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:39.402857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:39.402857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:39.402857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:39.402857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:39.402857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:39.402857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:39.402857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:39.402857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:39.402857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:39.402857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:39.402857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:39.402857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:48:39.402857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 56/119] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:39.403618 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e0223-554a-4ffd-8140-7f58f43d62f8 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:39.403618 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e0223-554a-4ffd-8140-7f58f43d62f8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:39.404142 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e0223-554a-4ffd-8140-7f58f43d62f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:39.410236 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3752a65-90c8-4dc6-b152-45aefcc7a8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:39.411193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3752a65-90c8-4dc6-b152-45aefcc7a8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:39.411473 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3752a65-90c8-4dc6-b152-45aefcc7a8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:39.411779 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3752a65-90c8-4dc6-b152-45aefcc7a8d0 None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:39.413005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3752a65-90c8-4dc6-b152-45aefcc7a8d0 None admin] RBAC: Authorizing `identity:get_project(project_id=b82ee754849c4779b408e856c429ab9b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:39.414468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3752a65-90c8-4dc6-b152-45aefcc7a8d0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:39.416213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 64/120] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:39 2022] GET /identity/v3/projects/b82ee754849c4779b408e856c429ab9b => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:39.416973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3752a65-90c8-4dc6-b152-45aefcc7a8d0 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:39.426811 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-234e0223-554a-4ffd-8140-7f58f43d62f8 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:39.428220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-234e0223-554a-4ffd-8140-7f58f43d62f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:39.436665 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:39.475077 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:39.502017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 57/121] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:39.509491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e0223-554a-4ffd-8140-7f58f43d62f8 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:39.529732 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3752a65-90c8-4dc6-b152-45aefcc7a8d0 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:39.568529 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26985847-82b4-4f52-88ef-73e19ad0b8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:39.569569 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26985847-82b4-4f52-88ef-73e19ad0b8bc None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:39.569929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26985847-82b4-4f52-88ef-73e19ad0b8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:39.570220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26985847-82b4-4f52-88ef-73e19ad0b8bc None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:39.571541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26985847-82b4-4f52-88ef-73e19ad0b8bc None admin] RBAC: Authorizing `identity:get_project(project_id=b82ee754849c4779b408e856c429ab9b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:39.573108 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26985847-82b4-4f52-88ef-73e19ad0b8bc None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:39.575094 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 65/122] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:39 2022] GET /identity/v3/projects/b82ee754849c4779b408e856c429ab9b => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:39.575800 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26985847-82b4-4f52-88ef-73e19ad0b8bc None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:39.628427 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e0223-554a-4ffd-8140-7f58f43d62f8 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:39.669236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fbf8c-395d-49b1-8b0a-fdc80c609296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:39.670263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fbf8c-395d-49b1-8b0a-fdc80c609296 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:39.670656 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fbf8c-395d-49b1-8b0a-fdc80c609296 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:39.671066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fbf8c-395d-49b1-8b0a-fdc80c609296 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:39.672496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168fbf8c-395d-49b1-8b0a-fdc80c609296 None admin] RBAC: Adding query filter params (name=member) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:39.672630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:39.673026 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168fbf8c-395d-49b1-8b0a-fdc80c609296 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:39.674326 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168fbf8c-395d-49b1-8b0a-fdc80c609296 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:39.686264 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 58/123] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:48:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:39.688806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26985847-82b4-4f52-88ef-73e19ad0b8bc None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:39.691350 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fbf8c-395d-49b1-8b0a-fdc80c609296 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:39.724619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f45b33-b168-41d4-b75c-63cf91521e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:39.725303 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f45b33-b168-41d4-b75c-63cf91521e74 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:39.725600 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f45b33-b168-41d4-b75c-63cf91521e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:39.725885 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f45b33-b168-41d4-b75c-63cf91521e74 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:39.726643 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f45b33-b168-41d4-b75c-63cf91521e74 None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:39.726962 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f45b33-b168-41d4-b75c-63cf91521e74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:39.728468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f45b33-b168-41d4-b75c-63cf91521e74 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:39.733046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:39.737182 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 66/124] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:48:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:39.743021 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f45b33-b168-41d4-b75c-63cf91521e74 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:39.823765 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fbf8c-395d-49b1-8b0a-fdc80c609296 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:39.863535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4617c-e075-4950-be56-e5cf16d571e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:39.863732 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f45b33-b168-41d4-b75c-63cf91521e74 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:39.864462 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4617c-e075-4950-be56-e5cf16d571e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:39.864749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4617c-e075-4950-be56-e5cf16d571e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:39.865034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4617c-e075-4950-be56-e5cf16d571e9 None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b/users/e383b5a76d134501a90ca526cde31f54/roles/394e165201dc41c8b3255de480821b1f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:39.867394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca4617c-e075-4950-be56-e5cf16d571e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=b82ee754849c4779b408e856c429ab9b, user_id=e383b5a76d134501a90ca526cde31f54, role_id=394e165201dc41c8b3255de480821b1f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:39.868867 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca4617c-e075-4950-be56-e5cf16d571e9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:39.897628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1281341-b541-437b-aff3-fa90286beb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:39.898427 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1281341-b541-437b-aff3-fa90286beb3d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:39.898732 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1281341-b541-437b-aff3-fa90286beb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:39.899003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1281341-b541-437b-aff3-fa90286beb3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:39.909134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1281341-b541-437b-aff3-fa90286beb3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:39.917547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-f1281341-b541-437b-aff3-fa90286beb3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:48:39.917547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:39.917547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:39.917547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:39.917547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:39.917547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:39.917547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:39.917547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:39.917547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:39.917547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:39.917547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:39.917547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:39.917547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:39.917547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:39.917547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:39.917547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:39.918580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:39.918580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:39.918580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:39.918580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:39.918580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:39.918580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:39.918580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:39.918580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:39.918580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:39.918580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:39.918580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:39.918580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:39.918580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:39.918580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:39.918580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:39.918580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:39.922930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:39.922930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:39.922930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:39.922930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:39.922930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:39.922930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:39.922930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:39.922930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:39.922930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:39.922930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:39.922930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:48:39.922930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:39.922930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 67/125] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:48:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:39.922930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1281341-b541-437b-aff3-fa90286beb3d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:39.925166 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:39.952409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 59/126] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:48:39 2022] PUT /identity/v3/projects/b82ee754849c4779b408e856c429ab9b/users/e383b5a76d134501a90ca526cde31f54/roles/394e165201dc41c8b3255de480821b1f => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:48:39.953684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4617c-e075-4950-be56-e5cf16d571e9 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:40.086733 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1281341-b541-437b-aff3-fa90286beb3d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:40.090536 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:40.098447 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4617c-e075-4950-be56-e5cf16d571e9 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:40.131898 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856ecf9-e90f-4c23-970b-73778a8f253e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:40.132212 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856ecf9-e90f-4c23-970b-73778a8f253e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:40.132491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856ecf9-e90f-4c23-970b-73778a8f253e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:40.132782 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856ecf9-e90f-4c23-970b-73778a8f253e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:40.133825 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856ecf9-e90f-4c23-970b-73778a8f253e None admin] RBAC: Adding query filter params (role.id=a2403691e5c440df989bc6aad7118272, scope.project.id=b82ee754849c4779b408e856c429ab9b, user.id=a2489171853b43e8bc7ff724ab503ed9) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:40.134168 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856ecf9-e90f-4c23-970b-73778a8f253e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:40.135675 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856ecf9-e90f-4c23-970b-73778a8f253e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:40.141731 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 68/127] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:48:39 2022] GET /identity/v3/role_assignments?user.id=a2489171853b43e8bc7ff724ab503ed9&scope.project.id=b82ee754849c4779b408e856c429ab9b&role.id=a2403691e5c440df989bc6aad7118272 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:40.142540 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856ecf9-e90f-4c23-970b-73778a8f253e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:40.146141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a4ce9-adf5-42ef-a055-be44f73f725a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:40.148136 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a4ce9-adf5-42ef-a055-be44f73f725a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:40.148433 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a4ce9-adf5-42ef-a055-be44f73f725a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:40.148720 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a4ce9-adf5-42ef-a055-be44f73f725a None admin] PATH_INFO: `/v3/users/e383b5a76d134501a90ca526cde31f54` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:40.152054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a4ce9-adf5-42ef-a055-be44f73f725a None admin] RBAC: Authorizing `identity:get_user(user_id=e383b5a76d134501a90ca526cde31f54)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:40.153874 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a4ce9-adf5-42ef-a055-be44f73f725a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:40.157280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 60/128] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:39 2022] GET /identity/v3/users/e383b5a76d134501a90ca526cde31f54 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:40.158431 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a4ce9-adf5-42ef-a055-be44f73f725a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:40.233538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:40.272370 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856ecf9-e90f-4c23-970b-73778a8f253e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:40.315093 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1402ab-0a80-4e75-a1c4-a4880b3b8d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:40.316043 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1402ab-0a80-4e75-a1c4-a4880b3b8d04 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:40.316339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1402ab-0a80-4e75-a1c4-a4880b3b8d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:40.316624 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1402ab-0a80-4e75-a1c4-a4880b3b8d04 None admin] PATH_INFO: `/v3/users/e383b5a76d134501a90ca526cde31f54` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:40.318549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1402ab-0a80-4e75-a1c4-a4880b3b8d04 None admin] RBAC: Authorizing `identity:get_user(user_id=e383b5a76d134501a90ca526cde31f54)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:40.320205 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1402ab-0a80-4e75-a1c4-a4880b3b8d04 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:40.327435 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 69/129] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:40 2022] GET /identity/v3/users/e383b5a76d134501a90ca526cde31f54 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:40.327435 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1402ab-0a80-4e75-a1c4-a4880b3b8d04 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:40.356653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a4ce9-adf5-42ef-a055-be44f73f725a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:40.403140 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7d982-0156-40fd-976a-a66e929746b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:40.404632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7d982-0156-40fd-976a-a66e929746b9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:40.405320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7d982-0156-40fd-976a-a66e929746b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:40.405755 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7d982-0156-40fd-976a-a66e929746b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:40.406826 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7d982-0156-40fd-976a-a66e929746b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:40.408115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7d982-0156-40fd-976a-a66e929746b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:40.409681 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7d982-0156-40fd-976a-a66e929746b9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:40.419649 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 61/130] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:40.420733 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7d982-0156-40fd-976a-a66e929746b9 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:40.458285 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:40.486041 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1402ab-0a80-4e75-a1c4-a4880b3b8d04 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:40.533782 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7d982-0156-40fd-976a-a66e929746b9 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:40.545780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2252613-e40c-4a20-b17f-2dc54b6c1e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:40.546789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2252613-e40c-4a20-b17f-2dc54b6c1e46 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:40.547199 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2252613-e40c-4a20-b17f-2dc54b6c1e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:40.547583 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2252613-e40c-4a20-b17f-2dc54b6c1e46 None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:40.548875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2252613-e40c-4a20-b17f-2dc54b6c1e46 None admin] RBAC: Authorizing `identity:get_project(project_id=b82ee754849c4779b408e856c429ab9b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:40.550441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2252613-e40c-4a20-b17f-2dc54b6c1e46 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:40.552877 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 70/131] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:40 2022] GET /identity/v3/projects/b82ee754849c4779b408e856c429ab9b => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:40.553653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2252613-e40c-4a20-b17f-2dc54b6c1e46 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:40.561808 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:40.573185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821c5d2-c43f-4dec-8277-c819b7926308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:40.574277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821c5d2-c43f-4dec-8277-c819b7926308 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:40.574713 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821c5d2-c43f-4dec-8277-c819b7926308 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:40.575274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821c5d2-c43f-4dec-8277-c819b7926308 None admin] PATH_INFO: `/v3/projects/0c022d6570d94db6874a209a32bedefe` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:40.576632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821c5d2-c43f-4dec-8277-c819b7926308 None admin] RBAC: Authorizing `identity:get_project(project_id=0c022d6570d94db6874a209a32bedefe)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:40.578548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821c5d2-c43f-4dec-8277-c819b7926308 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:40.580579 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 62/132] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:40 2022] GET /identity/v3/projects/0c022d6570d94db6874a209a32bedefe => generated 312 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:40.581931 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b55eeb-045d-468b-b042-8e4a5c8ed0fe None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:40.582379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b55eeb-045d-468b-b042-8e4a5c8ed0fe None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:40.582872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b55eeb-045d-468b-b042-8e4a5c8ed0fe None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:40.584382 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 63/133] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:40.585197 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b55eeb-045d-468b-b042-8e4a5c8ed0fe None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:40.693720 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2252613-e40c-4a20-b17f-2dc54b6c1e46 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:40.704353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:40.714583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b55eeb-045d-468b-b042-8e4a5c8ed0fe None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:40.734559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3bb9c7-92e7-445c-958f-575ac42da067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:40.735787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bb9c7-92e7-445c-958f-575ac42da067 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:40.736255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bb9c7-92e7-445c-958f-575ac42da067 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:40.736680 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bb9c7-92e7-445c-958f-575ac42da067 None admin] PATH_INFO: `/v3/users/82583283d3254dca9292c9de336ad74a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:40.738266 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3bb9c7-92e7-445c-958f-575ac42da067 None admin] RBAC: Authorizing `identity:get_user(user_id=82583283d3254dca9292c9de336ad74a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:40.740543 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3bb9c7-92e7-445c-958f-575ac42da067 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:40.742528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 71/134] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:40 2022] GET /identity/v3/users/82583283d3254dca9292c9de336ad74a => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:40.743731 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3bb9c7-92e7-445c-958f-575ac42da067 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:40.762787 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c448080-634a-4509-b196-e445f10be6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:40.764361 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c448080-634a-4509-b196-e445f10be6d7 None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:40.764932 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c448080-634a-4509-b196-e445f10be6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:40.765410 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c448080-634a-4509-b196-e445f10be6d7 None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b/users/e383b5a76d134501a90ca526cde31f54/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:40.767701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c448080-634a-4509-b196-e445f10be6d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b82ee754849c4779b408e856c429ab9b, user_id=e383b5a76d134501a90ca526cde31f54, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:40.769367 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c448080-634a-4509-b196-e445f10be6d7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:40.823285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:40.853507 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3bb9c7-92e7-445c-958f-575ac42da067 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:40.874984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 64/135] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:48:40 2022] PUT /identity/v3/projects/b82ee754849c4779b408e856c429ab9b/users/e383b5a76d134501a90ca526cde31f54/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:48:40.876879 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b05486-ffda-414d-b175-99ea50d9848e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:40.877832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b05486-ffda-414d-b175-99ea50d9848e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:40.878293 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b05486-ffda-414d-b175-99ea50d9848e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:40.894654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17818943-21a1-400c-b737-c4507b93509d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:40.895665 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17818943-21a1-400c-b737-c4507b93509d None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:40.896002 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17818943-21a1-400c-b737-c4507b93509d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:40.896290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17818943-21a1-400c-b737-c4507b93509d None admin] PATH_INFO: `/v3/projects/0c022d6570d94db6874a209a32bedefe/users/e383b5a76d134501a90ca526cde31f54/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:40.898657 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17818943-21a1-400c-b737-c4507b93509d None admin] RBAC: Authorizing `identity:create_grant(project_id=0c022d6570d94db6874a209a32bedefe, user_id=e383b5a76d134501a90ca526cde31f54, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:40.900259 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17818943-21a1-400c-b737-c4507b93509d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:40.902986 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-57b05486-ffda-414d-b175-99ea50d9848e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:40.906349 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-57b05486-ffda-414d-b175-99ea50d9848e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:40.994603 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 65/136] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:40.995498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b05486-ffda-414d-b175-99ea50d9848e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:40.996026 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 72/137] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:48:40 2022] PUT /identity/v3/projects/0c022d6570d94db6874a209a32bedefe/users/e383b5a76d134501a90ca526cde31f54/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:48:41.036814 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17818943-21a1-400c-b737-c4507b93509d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:41.103066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:41.121336 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b05486-ffda-414d-b175-99ea50d9848e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:41.171113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f8ae9-b2af-4959-a319-f91854b2872b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:41.171630 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f8ae9-b2af-4959-a319-f91854b2872b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:41.171922 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f8ae9-b2af-4959-a319-f91854b2872b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:41.172204 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f8ae9-b2af-4959-a319-f91854b2872b None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:41.174342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f8ae9-b2af-4959-a319-f91854b2872b None admin] RBAC: Authorizing `identity:get_project(project_id=b82ee754849c4779b408e856c429ab9b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:41.175800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f8ae9-b2af-4959-a319-f91854b2872b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:41.178185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 66/138] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:40 2022] GET /identity/v3/projects/b82ee754849c4779b408e856c429ab9b => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:41.184230 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f8ae9-b2af-4959-a319-f91854b2872b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:41.222361 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17818943-21a1-400c-b737-c4507b93509d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:41.284103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:41.292144 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210cf6d6-20d2-4564-8d25-421c3ebf0b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:41.298078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cf6d6-20d2-4564-8d25-421c3ebf0b5d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:41.298078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cf6d6-20d2-4564-8d25-421c3ebf0b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:41.298078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cf6d6-20d2-4564-8d25-421c3ebf0b5d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:41.298078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210cf6d6-20d2-4564-8d25-421c3ebf0b5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:41.298078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210cf6d6-20d2-4564-8d25-421c3ebf0b5d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:41.299184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 73/139] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:48:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:41.308228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210cf6d6-20d2-4564-8d25-421c3ebf0b5d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:41.347251 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f8ae9-b2af-4959-a319-f91854b2872b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:41.385509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54611f77-23b9-4230-8fb7-ac0c9565a490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:41.386850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54611f77-23b9-4230-8fb7-ac0c9565a490 None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:41.387360 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54611f77-23b9-4230-8fb7-ac0c9565a490 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:41.387838 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54611f77-23b9-4230-8fb7-ac0c9565a490 None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b/users/82583283d3254dca9292c9de336ad74a/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:41.390221 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54611f77-23b9-4230-8fb7-ac0c9565a490 None admin] RBAC: Authorizing `identity:create_grant(project_id=b82ee754849c4779b408e856c429ab9b, user_id=82583283d3254dca9292c9de336ad74a, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:41.392066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54611f77-23b9-4230-8fb7-ac0c9565a490 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:41.472555 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 67/140] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:48:41 2022] PUT /identity/v3/projects/b82ee754849c4779b408e856c429ab9b/users/82583283d3254dca9292c9de336ad74a/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 18:48:41.475774 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210cf6d6-20d2-4564-8d25-421c3ebf0b5d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:41.522516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d8e42-569e-4d2b-96f9-53bcb4f5ec4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:41.523593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d8e42-569e-4d2b-96f9-53bcb4f5ec4a None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:41.524076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d8e42-569e-4d2b-96f9-53bcb4f5ec4a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:41.524570 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d8e42-569e-4d2b-96f9-53bcb4f5ec4a None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:41.525534 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d8e42-569e-4d2b-96f9-53bcb4f5ec4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:41.527691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d8e42-569e-4d2b-96f9-53bcb4f5ec4a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:41.657097 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 74/141] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 14 18:48:41 2022] POST /identity/v3/users => generated 283 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:48:42.360058 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbea3b1-2823-4300-b8b5-c0fd1fe662e7 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:42.361061 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbea3b1-2823-4300-b8b5-c0fd1fe662e7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:42.361547 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbea3b1-2823-4300-b8b5-c0fd1fe662e7 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:42.363133 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 68/142] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:42.369087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b8c57-4780-4c1e-b02e-1adc4b6f9606 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:42.369887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b8c57-4780-4c1e-b02e-1adc4b6f9606 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:42.370388 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b8c57-4780-4c1e-b02e-1adc4b6f9606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:42.400387 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ec1b8c57-4780-4c1e-b02e-1adc4b6f9606 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:42.402149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ec1b8c57-4780-4c1e-b02e-1adc4b6f9606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:42.465788 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:42.483594 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 75/143] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:48:42.522768 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbea3b1-2823-4300-b8b5-c0fd1fe662e7 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:42.540387 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:42.643406 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbea3b1-2823-4300-b8b5-c0fd1fe662e7 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:42.681848 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76150510-f83c-44c6-a929-1318b7359322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:42.683008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76150510-f83c-44c6-a929-1318b7359322 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:42.683536 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76150510-f83c-44c6-a929-1318b7359322 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:42.684010 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76150510-f83c-44c6-a929-1318b7359322 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:42.700560 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76150510-f83c-44c6-a929-1318b7359322 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:42.709587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-76150510-f83c-44c6-a929-1318b7359322 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:48:42.709587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:42.709587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:42.709587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:42.709587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:42.709587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:42.709587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:42.709587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:42.709587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:42.709587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:42.709587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:42.709587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:42.709587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:42.709587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:42.709587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:42.709587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:42.710388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:42.710388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:42.710388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:42.710388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:42.710388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:42.710388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:42.710388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:42.710388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:42.710388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:42.710388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:42.710388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:42.710388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:42.710388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:42.710388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:42.710388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:42.710388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:42.711103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:42.711103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:42.711103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:42.711103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:42.711103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:42.711103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:42.711103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:42.711103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:42.711103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:42.711103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:42.711103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:48:42.711103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:48:42.713418 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 69/144] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:48:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:42.719350 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b8c57-4780-4c1e-b02e-1adc4b6f9606 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:42.833453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b8c57-4780-4c1e-b02e-1adc4b6f9606 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:42.878049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7269aa9-b406-4d45-97ff-9c3682be3070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:42.879098 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7269aa9-b406-4d45-97ff-9c3682be3070 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:42.879681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7269aa9-b406-4d45-97ff-9c3682be3070 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:42.880219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7269aa9-b406-4d45-97ff-9c3682be3070 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:42.881184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7269aa9-b406-4d45-97ff-9c3682be3070 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:42.881698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7269aa9-b406-4d45-97ff-9c3682be3070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:42.883375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7269aa9-b406-4d45-97ff-9c3682be3070 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:42.899603 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 76/145] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:48:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:42.905518 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76150510-f83c-44c6-a929-1318b7359322 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:43.007758 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:43.043903 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76150510-f83c-44c6-a929-1318b7359322 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:43.076519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6572f2b-9c74-41fb-af89-ed90c97ca7f6 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:43.077182 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6572f2b-9c74-41fb-af89-ed90c97ca7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:43.077666 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6572f2b-9c74-41fb-af89-ed90c97ca7f6 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:43.079367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 77/146] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:43.086577 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd15e4-e507-49ce-b569-de1ac5a3e221 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:43.087471 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd15e4-e507-49ce-b569-de1ac5a3e221 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:43.088175 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd15e4-e507-49ce-b569-de1ac5a3e221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:43.097873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e03fa8-753d-40e1-96c0-f4af37547674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:43.098832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e03fa8-753d-40e1-96c0-f4af37547674 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:43.099127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e03fa8-753d-40e1-96c0-f4af37547674 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:43.099440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e03fa8-753d-40e1-96c0-f4af37547674 None admin] PATH_INFO: `/v3/users/e383b5a76d134501a90ca526cde31f54` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:43.103102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e03fa8-753d-40e1-96c0-f4af37547674 None admin] RBAC: Authorizing `identity:get_user(user_id=e383b5a76d134501a90ca526cde31f54)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:43.104903 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e03fa8-753d-40e1-96c0-f4af37547674 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:43.106606 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 70/147] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:42 2022] GET /identity/v3/users/e383b5a76d134501a90ca526cde31f54 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:43.108906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5bfd15e4-e507-49ce-b569-de1ac5a3e221 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:43.110354 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5bfd15e4-e507-49ce-b569-de1ac5a3e221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:43.112243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e03fa8-753d-40e1-96c0-f4af37547674 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:43.220526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:43.292574 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 78/148] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:48:43.295740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e03fa8-753d-40e1-96c0-f4af37547674 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:43.328514 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd15e4-e507-49ce-b569-de1ac5a3e221 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:43.355907 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54f5a7-522e-4d39-8e02-22bca9c3ccf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:43.357074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54f5a7-522e-4d39-8e02-22bca9c3ccf5 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:43.357528 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54f5a7-522e-4d39-8e02-22bca9c3ccf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:43.357825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54f5a7-522e-4d39-8e02-22bca9c3ccf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:43.380077 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:43.482938 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54f5a7-522e-4d39-8e02-22bca9c3ccf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:43.484417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54f5a7-522e-4d39-8e02-22bca9c3ccf5 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:43.566960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd15e4-e507-49ce-b569-de1ac5a3e221 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:43.653286 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fb56c-a7c4-4009-a265-79362533e225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:43.654106 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fb56c-a7c4-4009-a265-79362533e225 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:43.654389 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fb56c-a7c4-4009-a265-79362533e225 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:43.654670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fb56c-a7c4-4009-a265-79362533e225 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:43.688157 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fb56c-a7c4-4009-a265-79362533e225 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:43.688860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 71/149] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:48:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 18:48:43.696668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-3a0fb56c-a7c4-4009-a265-79362533e225 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 18:48:43.696668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:43.696668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:43.696668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:43.696668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:43.696668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:43.696668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:43.696668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:43.696668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:43.696668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:43.696668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:43.696668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:43.696668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:43.696668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:43.696668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:43.696668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:43.697615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:43.697615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:43.697615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:43.697615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:43.697615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:43.697615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:43.697615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:43.697615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:43.697615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:43.697615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:43.697615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:43.697615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:43.697615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:43.697615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:43.697615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:43.697615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:43.698360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:43.698360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:43.698360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:43.698360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:43.698360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:43.698360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:43.698360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:43.698360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:43.698360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:43.698360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:43.698360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 18:48:43.698360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:43.698360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 79/150] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:48:43.704590 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54f5a7-522e-4d39-8e02-22bca9c3ccf5 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:43.710371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fb56c-a7c4-4009-a265-79362533e225 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:43.778494 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:43.811644 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:43.866320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54f5a7-522e-4d39-8e02-22bca9c3ccf5 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:43.927614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d9df2-e036-40d7-bfe9-1ddfb540ce0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:43.928880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d9df2-e036-40d7-bfe9-1ddfb540ce0a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:43.929368 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d9df2-e036-40d7-bfe9-1ddfb540ce0a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:43.929846 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d9df2-e036-40d7-bfe9-1ddfb540ce0a None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:43.933237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d9df2-e036-40d7-bfe9-1ddfb540ce0a None admin] RBAC: Authorizing `identity:get_project(project_id=b82ee754849c4779b408e856c429ab9b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:43.935256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d9df2-e036-40d7-bfe9-1ddfb540ce0a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:43.937692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 72/151] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:43 2022] GET /identity/v3/projects/b82ee754849c4779b408e856c429ab9b => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:43.944631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d9df2-e036-40d7-bfe9-1ddfb540ce0a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:43.974058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fb56c-a7c4-4009-a265-79362533e225 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:44.055992 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c04291-2eba-408d-bf3f-9602c088fe87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:44.056588 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c04291-2eba-408d-bf3f-9602c088fe87 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:44.056876 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c04291-2eba-408d-bf3f-9602c088fe87 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:44.057159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c04291-2eba-408d-bf3f-9602c088fe87 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:44.057923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c04291-2eba-408d-bf3f-9602c088fe87 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:44.058231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c04291-2eba-408d-bf3f-9602c088fe87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:44.059677 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c04291-2eba-408d-bf3f-9602c088fe87 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:44.069372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 80/152] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:48:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:44.078493 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c04291-2eba-408d-bf3f-9602c088fe87 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:44.121751 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d9df2-e036-40d7-bfe9-1ddfb540ce0a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:44.140263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:44.184130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c8ccc-3c99-467d-a6dd-cffec07556f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:44.185292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c8ccc-3c99-467d-a6dd-cffec07556f8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:44.185810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c8ccc-3c99-467d-a6dd-cffec07556f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:44.186335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c8ccc-3c99-467d-a6dd-cffec07556f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:44.187433 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c8ccc-3c99-467d-a6dd-cffec07556f8 None admin] RBAC: Adding query filter params (role.id=394e165201dc41c8b3255de480821b1f, scope.project.id=b82ee754849c4779b408e856c429ab9b, user.id=e383b5a76d134501a90ca526cde31f54) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:44.187983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c8ccc-3c99-467d-a6dd-cffec07556f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:44.189874 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c8ccc-3c99-467d-a6dd-cffec07556f8 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:44.201538 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 73/153] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:48:43 2022] GET /identity/v3/role_assignments?user.id=e383b5a76d134501a90ca526cde31f54&scope.project.id=b82ee754849c4779b408e856c429ab9b&role.id=394e165201dc41c8b3255de480821b1f => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:44.216318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:44.225705 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c04291-2eba-408d-bf3f-9602c088fe87 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:44.266227 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376b1d1-1187-43c4-93b1-d15b9684d116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:44.267118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376b1d1-1187-43c4-93b1-d15b9684d116 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:44.267426 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376b1d1-1187-43c4-93b1-d15b9684d116 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:44.267743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376b1d1-1187-43c4-93b1-d15b9684d116 None admin] PATH_INFO: `/v3/users/a2489171853b43e8bc7ff724ab503ed9` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:44.268923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376b1d1-1187-43c4-93b1-d15b9684d116 None admin] RBAC: Authorizing `identity:get_user(user_id=a2489171853b43e8bc7ff724ab503ed9)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:44.270443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376b1d1-1187-43c4-93b1-d15b9684d116 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:44.275355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 81/154] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:44 2022] GET /identity/v3/users/a2489171853b43e8bc7ff724ab503ed9 => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:44.279410 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c8ccc-3c99-467d-a6dd-cffec07556f8 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:44.417309 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bc2fc-d512-44dc-9808-9d633b5883eb None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:44.417625 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bc2fc-d512-44dc-9808-9d633b5883eb None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:44.417867 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bc2fc-d512-44dc-9808-9d633b5883eb None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:44.419493 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 82/155] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:44.424504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a5487-180f-4359-8175-76e5675ad639 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:44.424795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a5487-180f-4359-8175-76e5675ad639 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:44.425077 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a5487-180f-4359-8175-76e5675ad639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:44.444104 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-f55a5487-180f-4359-8175-76e5675ad639 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:44.445300 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f55a5487-180f-4359-8175-76e5675ad639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:44.477872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c8ccc-3c99-467d-a6dd-cffec07556f8 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:44.520090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 83/156] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:48:44.531954 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed492a35-0589-412e-8029-94ce5489e20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:44.533168 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed492a35-0589-412e-8029-94ce5489e20c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:44.533656 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed492a35-0589-412e-8029-94ce5489e20c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:44.534132 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed492a35-0589-412e-8029-94ce5489e20c None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:44.535571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed492a35-0589-412e-8029-94ce5489e20c None admin] RBAC: Authorizing `identity:get_project(project_id=b82ee754849c4779b408e856c429ab9b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:44.537860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed492a35-0589-412e-8029-94ce5489e20c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:44.540600 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 74/157] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:44 2022] GET /identity/v3/projects/b82ee754849c4779b408e856c429ab9b => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:44.543209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac47568-4729-4d0b-b996-d30039e1410c None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:44.543209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac47568-4729-4d0b-b996-d30039e1410c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:44.543209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac47568-4729-4d0b-b996-d30039e1410c None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:44.543209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 84/158] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:48:44.546698 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed492a35-0589-412e-8029-94ce5489e20c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:44.562626 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac47568-4729-4d0b-b996-d30039e1410c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:44.592657 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:44.677409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed492a35-0589-412e-8029-94ce5489e20c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:44.696493 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:44.701285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac47568-4729-4d0b-b996-d30039e1410c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:44.743542 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a074b-879a-4528-a8ce-33c223c5d633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:44.744345 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a074b-879a-4528-a8ce-33c223c5d633 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:44.744628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a074b-879a-4528-a8ce-33c223c5d633 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:44.744958 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a074b-879a-4528-a8ce-33c223c5d633 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:44.754848 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7d2bc-386d-4df6-ba05-08d28feae590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:44.756207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7d2bc-386d-4df6-ba05-08d28feae590 None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:44.756697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7d2bc-386d-4df6-ba05-08d28feae590 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:44.757173 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7d2bc-386d-4df6-ba05-08d28feae590 None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b/users/a2489171853b43e8bc7ff724ab503ed9/roles/a2403691e5c440df989bc6aad7118272` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:44.757393 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a074b-879a-4528-a8ce-33c223c5d633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:44.759689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e7d2bc-386d-4df6-ba05-08d28feae590 None admin] RBAC: Authorizing `identity:create_grant(project_id=b82ee754849c4779b408e856c429ab9b, user_id=a2489171853b43e8bc7ff724ab503ed9, role_id=a2403691e5c440df989bc6aad7118272)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:44.761442 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e7d2bc-386d-4df6-ba05-08d28feae590 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:44.767250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-850a074b-879a-4528-a8ce-33c223c5d633 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:44.767250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:44.767250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:44.767250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:44.767250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:44.767250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:44.767250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:44.767250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:44.767250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:44.767250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:44.767250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:44.767250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:44.767250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:44.767250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:44.767250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:44.767250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:44.768120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:44.768120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:44.768120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:44.768120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:44.768120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:44.768120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:44.768120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:44.768120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:44.768120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:44.768120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:44.768120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:44.768120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:44.768120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:44.768120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:44.768120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:44.768120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:44.768939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:44.768939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:44.768939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:44.768939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:44.768939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:44.768939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:44.768939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:44.768939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:44.768939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:44.768939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:44.768939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:44.768939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:44.768939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 85/159] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:44.768939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5486143-8a34-4b60-bd8c-be6c18d224b4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:44.769608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5486143-8a34-4b60-bd8c-be6c18d224b4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:44.769608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5486143-8a34-4b60-bd8c-be6c18d224b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:44.794965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-a5486143-8a34-4b60-bd8c-be6c18d224b4 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:44.796229 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a5486143-8a34-4b60-bd8c-be6c18d224b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:44.852574 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 75/160] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:48:44 2022] PUT /identity/v3/projects/b82ee754849c4779b408e856c429ab9b/users/a2489171853b43e8bc7ff724ab503ed9/roles/a2403691e5c440df989bc6aad7118272 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:48:44.853935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7d2bc-386d-4df6-ba05-08d28feae590 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:44.879222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 86/161] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:44.928597 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5486143-8a34-4b60-bd8c-be6c18d224b4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:45.076456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:45.082164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7d2bc-386d-4df6-ba05-08d28feae590 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:45.084236 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5486143-8a34-4b60-bd8c-be6c18d224b4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:45.087505 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:45.122398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f9255-55d2-4b7f-bcc4-928c4e1b56a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:45.123235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f9255-55d2-4b7f-bcc4-928c4e1b56a8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:45.123610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f9255-55d2-4b7f-bcc4-928c4e1b56a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:45.123890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f9255-55d2-4b7f-bcc4-928c4e1b56a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:45.136869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f9255-55d2-4b7f-bcc4-928c4e1b56a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:45.151209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-fd5f9255-55d2-4b7f-bcc4-928c4e1b56a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:45.151209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:45.151209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:45.151209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:45.151209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:45.151209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:45.151209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:45.151209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:45.151209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:45.151209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:45.151209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:45.151209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:45.151209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:45.151209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:45.151209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:45.151209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:45.152158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:45.152158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:45.152158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:45.152158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:45.152158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:45.152158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:45.152158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:45.152158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:45.152158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:45.152158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:45.152158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:45.152158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:45.152158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:45.152158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:45.152158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:45.152158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:45.153120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:45.153120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:45.153120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:45.153120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:45.153120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:45.153120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:45.153120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:45.153120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:45.153120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:45.153120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:45.153120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:45.153120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:45.153120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 87/162] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:48:45.153120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d228d77-6a95-4edb-9d89-4915f14960ae None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:45.153870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d228d77-6a95-4edb-9d89-4915f14960ae None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:45.153870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d228d77-6a95-4edb-9d89-4915f14960ae None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:45.153870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 88/163] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:45.155765 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d228d77-6a95-4edb-9d89-4915f14960ae None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:45.181332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e9ec4-cc9f-46a2-a859-9711b26330e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:45.183336 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e9ec4-cc9f-46a2-a859-9711b26330e0 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:45.184069 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e9ec4-cc9f-46a2-a859-9711b26330e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:45.184497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e9ec4-cc9f-46a2-a859-9711b26330e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:45.185397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e9ec4-cc9f-46a2-a859-9711b26330e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:45.185936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e9ec4-cc9f-46a2-a859-9711b26330e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:45.187505 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e9ec4-cc9f-46a2-a859-9711b26330e0 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:45.205827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 76/164] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:48:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:45.210190 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d19845-ad46-4759-a77a-84f747cb780e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:45.215523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d19845-ad46-4759-a77a-84f747cb780e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:45.216012 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d19845-ad46-4759-a77a-84f747cb780e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:45.252438 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-04d19845-ad46-4759-a77a-84f747cb780e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:45.254073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-04d19845-ad46-4759-a77a-84f747cb780e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:45.366304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d228d77-6a95-4edb-9d89-4915f14960ae None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:45.395832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 77/165] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:45.397062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d19845-ad46-4759-a77a-84f747cb780e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:45.450909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e04ffd-29b6-4542-ab03-c6ba95b9e713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:45.451719 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e04ffd-29b6-4542-ab03-c6ba95b9e713 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:45.452111 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e04ffd-29b6-4542-ab03-c6ba95b9e713 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:45.452492 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e04ffd-29b6-4542-ab03-c6ba95b9e713 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:45.453371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e04ffd-29b6-4542-ab03-c6ba95b9e713 None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:45.453786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e04ffd-29b6-4542-ab03-c6ba95b9e713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:45.455618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e04ffd-29b6-4542-ab03-c6ba95b9e713 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:45.478559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 89/166] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:48:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:45.479451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e04ffd-29b6-4542-ab03-c6ba95b9e713 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:45.495663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:45.549972 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:45.558969 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d19845-ad46-4759-a77a-84f747cb780e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:45.595964 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e2346-5e45-47f9-9438-36145ef87c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:45.596853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e2346-5e45-47f9-9438-36145ef87c28 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:45.597138 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e2346-5e45-47f9-9438-36145ef87c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:45.597415 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e2346-5e45-47f9-9438-36145ef87c28 None admin] PATH_INFO: `/v3/users/e383b5a76d134501a90ca526cde31f54` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:45.598740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e2346-5e45-47f9-9438-36145ef87c28 None admin] RBAC: Authorizing `identity:get_user(user_id=e383b5a76d134501a90ca526cde31f54)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:45.600246 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e2346-5e45-47f9-9438-36145ef87c28 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:45.601898 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 78/167] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:45 2022] GET /identity/v3/users/e383b5a76d134501a90ca526cde31f54 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:45.602658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e2346-5e45-47f9-9438-36145ef87c28 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:45.665991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e04ffd-29b6-4542-ab03-c6ba95b9e713 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:45.710257 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e2346-5e45-47f9-9438-36145ef87c28 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:45.746312 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50a920-6200-49ce-bdb8-1fa3793ded9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:45.747237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50a920-6200-49ce-bdb8-1fa3793ded9a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:45.747532 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50a920-6200-49ce-bdb8-1fa3793ded9a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:45.747814 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50a920-6200-49ce-bdb8-1fa3793ded9a None admin] PATH_INFO: `/v3/users/e383b5a76d134501a90ca526cde31f54` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:45.749028 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50a920-6200-49ce-bdb8-1fa3793ded9a None admin] RBAC: Authorizing `identity:get_user(user_id=e383b5a76d134501a90ca526cde31f54)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:45.750491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50a920-6200-49ce-bdb8-1fa3793ded9a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:45.752172 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 79/168] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:45 2022] GET /identity/v3/users/e383b5a76d134501a90ca526cde31f54 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:45.752947 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50a920-6200-49ce-bdb8-1fa3793ded9a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:45.764482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447db2a-6a46-46a8-a93b-6aa161bcff7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:45.765314 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447db2a-6a46-46a8-a93b-6aa161bcff7a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:45.765604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447db2a-6a46-46a8-a93b-6aa161bcff7a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:45.765880 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447db2a-6a46-46a8-a93b-6aa161bcff7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:45.799492 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:45.810674 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447db2a-6a46-46a8-a93b-6aa161bcff7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:45.819123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-b447db2a-6a46-46a8-a93b-6aa161bcff7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:48:45.819123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:45.819123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:45.819123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:45.819123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:45.819123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:45.819123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:45.819123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:45.819123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:45.819123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:45.819123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:45.819123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:45.819123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:45.819123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:45.819123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:45.819123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:45.820025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:45.820025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:45.820025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:45.820025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:45.820025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:45.820025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:45.820025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:45.820025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:45.820025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:45.820025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:45.820025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:45.820025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:45.820025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:45.820025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:45.820025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:45.820025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:45.820786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:45.820786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:45.820786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:45.820786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:45.820786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:45.820786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:45.820786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:45.820786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:45.820786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:45.820786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:45.820786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:48:45.820786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:45.824494 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 90/169] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:48:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:45.825549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447db2a-6a46-46a8-a93b-6aa161bcff7a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:45.870404 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50a920-6200-49ce-bdb8-1fa3793ded9a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:45.912897 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e6c4b-e15a-4a56-b26c-d862e69c8e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:45.913566 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e6c4b-e15a-4a56-b26c-d862e69c8e0e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:45.913861 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e6c4b-e15a-4a56-b26c-d862e69c8e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:45.914152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e6c4b-e15a-4a56-b26c-d862e69c8e0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:45.965500 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:46.035586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447db2a-6a46-46a8-a93b-6aa161bcff7a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:46.063509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e6c4b-e15a-4a56-b26c-d862e69c8e0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:46.064999 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e6c4b-e15a-4a56-b26c-d862e69c8e0e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:46.134448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:46.147898 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14279203-3574-4661-bbd7-f6c3d9f99180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:46.151863 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14279203-3574-4661-bbd7-f6c3d9f99180 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:46.151863 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14279203-3574-4661-bbd7-f6c3d9f99180 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:46.151863 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14279203-3574-4661-bbd7-f6c3d9f99180 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:46.284075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 80/170] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:48:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:48:46.284853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e6c4b-e15a-4a56-b26c-d862e69c8e0e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:46.320307 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14279203-3574-4661-bbd7-f6c3d9f99180 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:46.321892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14279203-3574-4661-bbd7-f6c3d9f99180 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:46.475945 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:46.505315 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e6c4b-e15a-4a56-b26c-d862e69c8e0e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:46.513439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 91/171] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:48:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 688 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:48:46.514653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14279203-3574-4661-bbd7-f6c3d9f99180 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:46.559338 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:46.571097 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69f2a6-52da-4360-bb1d-07192c1995a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:46.573302 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69f2a6-52da-4360-bb1d-07192c1995a2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:46.573302 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69f2a6-52da-4360-bb1d-07192c1995a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:46.573302 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69f2a6-52da-4360-bb1d-07192c1995a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:46.573302 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69f2a6-52da-4360-bb1d-07192c1995a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:46.573588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69f2a6-52da-4360-bb1d-07192c1995a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:46.575706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69f2a6-52da-4360-bb1d-07192c1995a2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:46.588670 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 81/172] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:46.590344 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65d7a4-56a3-4161-af94-0bc5e74246fc None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:46.590851 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65d7a4-56a3-4161-af94-0bc5e74246fc None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:46.591438 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65d7a4-56a3-4161-af94-0bc5e74246fc None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:46.593235 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 82/173] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:46.597413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65d7a4-56a3-4161-af94-0bc5e74246fc None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:46.653583 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14279203-3574-4661-bbd7-f6c3d9f99180 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:46.700360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcf93f-0cf9-4b63-ba2c-7a8ed80159c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:46.701822 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcf93f-0cf9-4b63-ba2c-7a8ed80159c6 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:46.702397 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcf93f-0cf9-4b63-ba2c-7a8ed80159c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:46.702875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcf93f-0cf9-4b63-ba2c-7a8ed80159c6 None admin] PATH_INFO: `/v3/projects/0c022d6570d94db6874a209a32bedefe` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:46.704723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcf93f-0cf9-4b63-ba2c-7a8ed80159c6 None admin] RBAC: Authorizing `identity:get_project(project_id=0c022d6570d94db6874a209a32bedefe)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:46.706224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcf93f-0cf9-4b63-ba2c-7a8ed80159c6 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:46.712586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 92/174] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:46 2022] GET /identity/v3/projects/0c022d6570d94db6874a209a32bedefe => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:46.712586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866aa40-74f1-49d0-baa9-290ce1fbabe5 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:46.712586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866aa40-74f1-49d0-baa9-290ce1fbabe5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:46.712586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866aa40-74f1-49d0-baa9-290ce1fbabe5 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:46.712586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 93/175] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:46.712586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866aa40-74f1-49d0-baa9-290ce1fbabe5 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:46.722269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65d7a4-56a3-4161-af94-0bc5e74246fc None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:46.755310 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:46.761432 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0bf8a-3a49-4fc6-9e95-1dd5fb826e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:46.762304 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0bf8a-3a49-4fc6-9e95-1dd5fb826e8b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:46.762593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0bf8a-3a49-4fc6-9e95-1dd5fb826e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:46.762895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0bf8a-3a49-4fc6-9e95-1dd5fb826e8b None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:46.764116 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0bf8a-3a49-4fc6-9e95-1dd5fb826e8b None admin] RBAC: Authorizing `identity:get_project(project_id=b82ee754849c4779b408e856c429ab9b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:46.765626 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0bf8a-3a49-4fc6-9e95-1dd5fb826e8b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:46.770712 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 83/176] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:46 2022] GET /identity/v3/projects/b82ee754849c4779b408e856c429ab9b => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:46.770712 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6902120-7ebf-4d9a-ac42-1c88c679bbb4 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:46.770712 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6902120-7ebf-4d9a-ac42-1c88c679bbb4 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:46.770712 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6902120-7ebf-4d9a-ac42-1c88c679bbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:46.788738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-e6902120-7ebf-4d9a-ac42-1c88c679bbb4 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:46.790004 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e6902120-7ebf-4d9a-ac42-1c88c679bbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:46.826383 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866aa40-74f1-49d0-baa9-290ce1fbabe5 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:46.848543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:46.862117 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 84/177] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:46.863016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5eeb5e-a448-4705-be10-11867b8456b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:46.863915 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5eeb5e-a448-4705-be10-11867b8456b4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:46.864205 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5eeb5e-a448-4705-be10-11867b8456b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:46.864485 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5eeb5e-a448-4705-be10-11867b8456b4 None admin] PATH_INFO: `/v3/users/82583283d3254dca9292c9de336ad74a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:46.865745 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5eeb5e-a448-4705-be10-11867b8456b4 None admin] RBAC: Authorizing `identity:get_user(user_id=82583283d3254dca9292c9de336ad74a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:46.867247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5eeb5e-a448-4705-be10-11867b8456b4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:46.869003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 94/178] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:46 2022] GET /identity/v3/users/82583283d3254dca9292c9de336ad74a => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:46.879192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043d4d6-ad4b-4289-8a80-cd24945ed6ab None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:46.879192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043d4d6-ad4b-4289-8a80-cd24945ed6ab None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:46.879192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043d4d6-ad4b-4289-8a80-cd24945ed6ab None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:46.880089 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5eeb5e-a448-4705-be10-11867b8456b4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:46.880184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 85/179] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:46 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:46.880184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adbeb4-0247-4cf6-9f0d-82a26c9a1c68 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:46.880184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adbeb4-0247-4cf6-9f0d-82a26c9a1c68 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:46.880184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adbeb4-0247-4cf6-9f0d-82a26c9a1c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:46.894072 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-45adbeb4-0247-4cf6-9f0d-82a26c9a1c68 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:46.895901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-45adbeb4-0247-4cf6-9f0d-82a26c9a1c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:46.969168 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 86/180] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:46.976204 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adbeb4-0247-4cf6-9f0d-82a26c9a1c68 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:46.989920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5eeb5e-a448-4705-be10-11867b8456b4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:46.999412 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:47.029055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6b36e-c72c-4055-a236-9cdf51df6562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:47.029720 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6b36e-c72c-4055-a236-9cdf51df6562 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:47.030002 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6b36e-c72c-4055-a236-9cdf51df6562 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:47.030283 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6b36e-c72c-4055-a236-9cdf51df6562 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:47.031115 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6b36e-c72c-4055-a236-9cdf51df6562 None admin] RBAC: Adding query filter params (role.id=ffed4493824a4040977d2cd1d057e07b, scope.project.id=0c022d6570d94db6874a209a32bedefe, user.id=e383b5a76d134501a90ca526cde31f54) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:47.031452 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6b36e-c72c-4055-a236-9cdf51df6562 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:47.033024 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6b36e-c72c-4055-a236-9cdf51df6562 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:47.040428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 95/181] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:48:46 2022] GET /identity/v3/role_assignments?user.id=e383b5a76d134501a90ca526cde31f54&scope.project.id=0c022d6570d94db6874a209a32bedefe&role.id=ffed4493824a4040977d2cd1d057e07b => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:47.041226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6b36e-c72c-4055-a236-9cdf51df6562 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:47.102422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adbeb4-0247-4cf6-9f0d-82a26c9a1c68 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:47.132514 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:47.163501 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705e937-f59d-48bd-8ccc-b0a84eb1710d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:47.164180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705e937-f59d-48bd-8ccc-b0a84eb1710d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:47.164464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705e937-f59d-48bd-8ccc-b0a84eb1710d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:47.164793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705e937-f59d-48bd-8ccc-b0a84eb1710d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:47.165578 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705e937-f59d-48bd-8ccc-b0a84eb1710d None admin] RBAC: Adding query filter params (role.id=ffed4493824a4040977d2cd1d057e07b, scope.project.id=b82ee754849c4779b408e856c429ab9b, user.id=e383b5a76d134501a90ca526cde31f54) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:47.165889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705e937-f59d-48bd-8ccc-b0a84eb1710d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:47.167380 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705e937-f59d-48bd-8ccc-b0a84eb1710d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:47.167773 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6b36e-c72c-4055-a236-9cdf51df6562 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:47.178421 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 87/182] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:48:46 2022] GET /identity/v3/role_assignments?user.id=e383b5a76d134501a90ca526cde31f54&scope.project.id=b82ee754849c4779b408e856c429ab9b&role.id=ffed4493824a4040977d2cd1d057e07b => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:47.178421 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601689e-1a0e-4897-b763-c286190b4b3f None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:47.178421 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601689e-1a0e-4897-b763-c286190b4b3f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:47.178421 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601689e-1a0e-4897-b763-c286190b4b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:47.212755 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-8601689e-1a0e-4897-b763-c286190b4b3f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:47.214209 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8601689e-1a0e-4897-b763-c286190b4b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:47.222685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2975c-0777-404c-89d9-862a6e751d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:47.224352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2975c-0777-404c-89d9-862a6e751d53 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:47.224794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2975c-0777-404c-89d9-862a6e751d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:47.225217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2975c-0777-404c-89d9-862a6e751d53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:47.291603 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:47.312678 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 88/183] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:47.316848 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601689e-1a0e-4897-b763-c286190b4b3f None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:47.326893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2975c-0777-404c-89d9-862a6e751d53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:47.328815 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2975c-0777-404c-89d9-862a6e751d53 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:47.450086 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601689e-1a0e-4897-b763-c286190b4b3f None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:47.466207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:47.482573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 96/184] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:48:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:48:47.485768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2975c-0777-404c-89d9-862a6e751d53 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:47.494799 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45e1f8-e75a-46d4-986e-44c9ec00ffea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:47.501391 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45e1f8-e75a-46d4-986e-44c9ec00ffea None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:47.501391 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45e1f8-e75a-46d4-986e-44c9ec00ffea None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:47.501391 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45e1f8-e75a-46d4-986e-44c9ec00ffea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:47.509673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e45e1f8-e75a-46d4-986e-44c9ec00ffea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:47.517876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-5e45e1f8-e75a-46d4-986e-44c9ec00ffea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:48:47.517876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:47.517876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:47.517876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:47.517876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:47.517876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:47.517876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:47.517876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:47.517876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:47.517876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:47.517876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:47.517876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:47.517876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:47.517876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:47.517876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:47.517876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:47.518715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:47.518715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:47.518715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:47.518715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:47.518715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:47.518715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:47.518715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:47.518715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:47.518715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:47.518715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:47.518715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:47.518715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:47.518715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:47.518715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:47.518715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:47.518715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:47.519575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:47.519575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:47.519575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:47.519575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:47.519575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:47.519575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:47.519575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:47.519575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:47.519575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:47.519575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:47.519575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:48:47.519575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:48:47.519575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 89/185] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:48:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:47.520762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45e1f8-e75a-46d4-986e-44c9ec00ffea None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:47.569051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:47.600959 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2975c-0777-404c-89d9-862a6e751d53 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:47.627656 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45e1f8-e75a-46d4-986e-44c9ec00ffea None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:47.637327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3662b-cc92-47b5-95db-55d7b7178c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:47.638108 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3662b-cc92-47b5-95db-55d7b7178c29 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:47.638400 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3662b-cc92-47b5-95db-55d7b7178c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:47.638672 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3662b-cc92-47b5-95db-55d7b7178c29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:47.640002 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3662b-cc92-47b5-95db-55d7b7178c29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:47.641447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3662b-cc92-47b5-95db-55d7b7178c29 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:47.643071 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 97/186] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:48:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:47.643843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3662b-cc92-47b5-95db-55d7b7178c29 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:47.664213 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ab443-1bb9-4685-b884-6077c1855ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:47.665019 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ab443-1bb9-4685-b884-6077c1855ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:47.665303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ab443-1bb9-4685-b884-6077c1855ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:47.665578 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ab443-1bb9-4685-b884-6077c1855ef7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:47.675623 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ab443-1bb9-4685-b884-6077c1855ef7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:47.684356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-728ab443-1bb9-4685-b884-6077c1855ef7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:48:47.684356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:47.684356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:47.684356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:47.684356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:47.684356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:47.684356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:47.684356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:47.684356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:47.684356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:47.684356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:47.684356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:47.684356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:47.684356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:47.684356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:47.684356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:47.685192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:47.685192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:47.685192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:47.685192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:47.685192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:47.685192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:47.685192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:47.685192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:47.685192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:47.685192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:47.685192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:47.685192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:47.685192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:47.685192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:47.685192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:47.685192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:47.685953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:47.685953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:47.685953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:47.685953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:47.685953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:47.685953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:47.685953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:47.685953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:47.685953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:47.685953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:47.685953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:48:47.685953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:48:47.685953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 90/187] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:48:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:47.686598 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ab443-1bb9-4685-b884-6077c1855ef7 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:47.720673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:47.742429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3662b-cc92-47b5-95db-55d7b7178c29 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:47.774907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8febd6-ec8a-471c-9dc6-1948aa5ec302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:47.775787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8febd6-ec8a-471c-9dc6-1948aa5ec302 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:47.776337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8febd6-ec8a-471c-9dc6-1948aa5ec302 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:47.776611 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8febd6-ec8a-471c-9dc6-1948aa5ec302 None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:47.777780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8febd6-ec8a-471c-9dc6-1948aa5ec302 None admin] RBAC: Authorizing `identity:get_project(project_id=b82ee754849c4779b408e856c429ab9b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:47.779259 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8febd6-ec8a-471c-9dc6-1948aa5ec302 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:47.780944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 98/188] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:47 2022] GET /identity/v3/projects/b82ee754849c4779b408e856c429ab9b => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:47.782104 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acd7a5-6701-4f8b-998a-a9f97da4357b None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:47.782386 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acd7a5-6701-4f8b-998a-a9f97da4357b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:47.782662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acd7a5-6701-4f8b-998a-a9f97da4357b None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:47.783908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 99/189] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:47.788892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39acd7a5-6701-4f8b-998a-a9f97da4357b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:47.793158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ab443-1bb9-4685-b884-6077c1855ef7 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:47.794729 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:47.826694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943612d-46c6-4d07-a162-0d9e8703af8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:47.827393 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943612d-46c6-4d07-a162-0d9e8703af8b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:47.827680 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943612d-46c6-4d07-a162-0d9e8703af8b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:47.827959 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943612d-46c6-4d07-a162-0d9e8703af8b None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:47.828700 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943612d-46c6-4d07-a162-0d9e8703af8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:47.829008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943612d-46c6-4d07-a162-0d9e8703af8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:47.830384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943612d-46c6-4d07-a162-0d9e8703af8b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:47.838329 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 91/190] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:47.839104 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943612d-46c6-4d07-a162-0d9e8703af8b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:47.889951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39acd7a5-6701-4f8b-998a-a9f97da4357b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:47.922958 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a973654-daf5-4504-ad91-1a6f93bb8ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:47.923650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a973654-daf5-4504-ad91-1a6f93bb8ccd None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:47.923935 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a973654-daf5-4504-ad91-1a6f93bb8ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:47.924213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a973654-daf5-4504-ad91-1a6f93bb8ccd None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:47.924918 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a973654-daf5-4504-ad91-1a6f93bb8ccd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:47.926427 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a973654-daf5-4504-ad91-1a6f93bb8ccd None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:47.932295 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:47.939239 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943612d-46c6-4d07-a162-0d9e8703af8b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:47.972055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566495f-5525-411f-9c87-4f4d4b741633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:47.972649 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566495f-5525-411f-9c87-4f4d4b741633 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:47.972933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566495f-5525-411f-9c87-4f4d4b741633 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:47.973819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566495f-5525-411f-9c87-4f4d4b741633 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:47.974572 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566495f-5525-411f-9c87-4f4d4b741633 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:47.974880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566495f-5525-411f-9c87-4f4d4b741633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:47.976292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566495f-5525-411f-9c87-4f4d4b741633 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:47.984003 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 92/191] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:48:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:47.987523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566495f-5525-411f-9c87-4f4d4b741633 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:48.004412 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 100/192] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 14 18:48:47 2022] POST /identity/v3/users => generated 297 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:48:48.028799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbe155-f529-4835-a047-5a120f78bb65 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:48.029155 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbe155-f529-4835-a047-5a120f78bb65 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:48.029367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbe155-f529-4835-a047-5a120f78bb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:48.047893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7bcbe155-f529-4835-a047-5a120f78bb65 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:48.049130 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7bcbe155-f529-4835-a047-5a120f78bb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:48.108971 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566495f-5525-411f-9c87-4f4d4b741633 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:48.120000 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:48.131068 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 101/193] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:48.132269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcbe155-f529-4835-a047-5a120f78bb65 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:48.163096 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc7a0a-4449-4a62-a1c0-663c99287fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:48.163096 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc7a0a-4449-4a62-a1c0-663c99287fae None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:48.167102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc7a0a-4449-4a62-a1c0-663c99287fae None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:48.167102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc7a0a-4449-4a62-a1c0-663c99287fae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:48.167102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc7a0a-4449-4a62-a1c0-663c99287fae None admin] RBAC: Adding query filter params (role.id=27fc14631c494ca0b2dc7f708be016c2, scope.project.id=b82ee754849c4779b408e856c429ab9b, user.id=82583283d3254dca9292c9de336ad74a) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:48.167102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc7a0a-4449-4a62-a1c0-663c99287fae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:48.167102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc7a0a-4449-4a62-a1c0-663c99287fae None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:48.172595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 93/194] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:48:47 2022] GET /identity/v3/role_assignments?user.id=82583283d3254dca9292c9de336ad74a&scope.project.id=b82ee754849c4779b408e856c429ab9b&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:48.173476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc7a0a-4449-4a62-a1c0-663c99287fae None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:48.198752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:48.271246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcbe155-f529-4835-a047-5a120f78bb65 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:48.316385 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc7a0a-4449-4a62-a1c0-663c99287fae None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:48.322288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3febf-442b-4ee1-a7b1-189da3a7f82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:48.323214 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3febf-442b-4ee1-a7b1-189da3a7f82c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:48.323510 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3febf-442b-4ee1-a7b1-189da3a7f82c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:48.323789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3febf-442b-4ee1-a7b1-189da3a7f82c None admin] PATH_INFO: `/v3/users/5c93b65311c34f30bf0f65e27782e21d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:48.343304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e3febf-442b-4ee1-a7b1-189da3a7f82c None admin] RBAC: Authorizing `identity:get_user(user_id=5c93b65311c34f30bf0f65e27782e21d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:48.344809 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e3febf-442b-4ee1-a7b1-189da3a7f82c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:48.346729 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 102/195] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:48 2022] GET /identity/v3/users/5c93b65311c34f30bf0f65e27782e21d => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:48.347364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3febf-442b-4ee1-a7b1-189da3a7f82c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:48.374693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d948ee2-36ce-4012-9503-ceca3abba9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:48.375611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d948ee2-36ce-4012-9503-ceca3abba9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:48.375893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d948ee2-36ce-4012-9503-ceca3abba9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:48.376169 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d948ee2-36ce-4012-9503-ceca3abba9a9 None admin] PATH_INFO: `/v3/users/5c93b65311c34f30bf0f65e27782e21d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:48.377391 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d948ee2-36ce-4012-9503-ceca3abba9a9 None admin] RBAC: Authorizing `identity:get_user(user_id=5c93b65311c34f30bf0f65e27782e21d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:48.378855 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d948ee2-36ce-4012-9503-ceca3abba9a9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:48.381387 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 94/196] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:48 2022] GET /identity/v3/users/5c93b65311c34f30bf0f65e27782e21d => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:48.382175 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d948ee2-36ce-4012-9503-ceca3abba9a9 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:48.418843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:48.505461 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:48.514888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3febf-442b-4ee1-a7b1-189da3a7f82c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:48.518194 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d948ee2-36ce-4012-9503-ceca3abba9a9 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:48.554561 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0e5e7-37c2-484c-8d3f-dac1c0705a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:48.555321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0e5e7-37c2-484c-8d3f-dac1c0705a68 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:48.555604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0e5e7-37c2-484c-8d3f-dac1c0705a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:48.555704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b93a8-eac9-4857-8883-a8a0bb67484a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:48.555882 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0e5e7-37c2-484c-8d3f-dac1c0705a68 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:48.556414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b93a8-eac9-4857-8883-a8a0bb67484a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:48.556711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b93a8-eac9-4857-8883-a8a0bb67484a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:48.556997 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b93a8-eac9-4857-8883-a8a0bb67484a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:48.569949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0e5e7-37c2-484c-8d3f-dac1c0705a68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:48.577866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-23e0e5e7-37c2-484c-8d3f-dac1c0705a68 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 18:48:48.577866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:48.577866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:48.577866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:48.577866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:48.577866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:48.577866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:48.577866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:48.577866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:48.577866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:48.577866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:48.577866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:48.577866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:48.577866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:48.577866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:48.577866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:48.578740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:48.578740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:48.578740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:48.578740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:48.578740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:48.578740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:48.578740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:48.578740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:48.578740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:48.578740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:48.578740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:48.578740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:48.578740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:48.578740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:48.578740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:48.578740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:48.579582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:48.579582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:48.579582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:48.579582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:48.579582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:48.579582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:48.579582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:48.579582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:48.579582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:48.579582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:48.579582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 18:48:48.579582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:48.581600 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 103/197] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:48.582438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0e5e7-37c2-484c-8d3f-dac1c0705a68 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:48.635676 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b93a8-eac9-4857-8883-a8a0bb67484a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:48.637160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b93a8-eac9-4857-8883-a8a0bb67484a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:48.700906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0e5e7-37c2-484c-8d3f-dac1c0705a68 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:48.715577 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:48.735979 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:48.737399 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee72871d-a57b-4821-b4e2-cb9592038613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:48.738057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72871d-a57b-4821-b4e2-cb9592038613 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:48.738396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72871d-a57b-4821-b4e2-cb9592038613 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:48.738684 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72871d-a57b-4821-b4e2-cb9592038613 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:48.750155 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 95/198] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:48:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:48:48.751247 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b93a8-eac9-4857-8883-a8a0bb67484a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:48.833189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee72871d-a57b-4821-b4e2-cb9592038613 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:48.835183 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee72871d-a57b-4821-b4e2-cb9592038613 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:48.869544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b93a8-eac9-4857-8883-a8a0bb67484a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:48.910316 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9fead-da91-4569-ac2e-44914bde848c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:48.911584 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9fead-da91-4569-ac2e-44914bde848c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:48.911875 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9fead-da91-4569-ac2e-44914bde848c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:48.912156 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9fead-da91-4569-ac2e-44914bde848c None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:48.913006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9fead-da91-4569-ac2e-44914bde848c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:48.913311 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9fead-da91-4569-ac2e-44914bde848c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:48.914705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9fead-da91-4569-ac2e-44914bde848c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:48.923203 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 96/199] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:48:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:48.924427 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9fead-da91-4569-ac2e-44914bde848c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:48.945537 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 104/200] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:48:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:48:48.946498 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee72871d-a57b-4821-b4e2-cb9592038613 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:48.959719 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:48.980344 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:49.038098 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9fead-da91-4569-ac2e-44914bde848c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:49.061219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee72871d-a57b-4821-b4e2-cb9592038613 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:49.073827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0995d-207d-43ff-8b13-da0a8cb59e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:49.074744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0995d-207d-43ff-8b13-da0a8cb59e74 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:49.074958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0995d-207d-43ff-8b13-da0a8cb59e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:49.075256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0995d-207d-43ff-8b13-da0a8cb59e74 None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:49.076434 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0995d-207d-43ff-8b13-da0a8cb59e74 None admin] RBAC: Authorizing `identity:get_project(project_id=45bac146c9d842bf82503d3f2b145128)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:49.078050 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0995d-207d-43ff-8b13-da0a8cb59e74 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:49.079923 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 97/201] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:48 2022] GET /identity/v3/projects/45bac146c9d842bf82503d3f2b145128 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:49.080770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0995d-207d-43ff-8b13-da0a8cb59e74 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:49.100833 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fa1a7-6e1a-49fd-ad8b-19fee82ccac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:49.101912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fa1a7-6e1a-49fd-ad8b-19fee82ccac9 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:49.102344 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fa1a7-6e1a-49fd-ad8b-19fee82ccac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:49.102878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fa1a7-6e1a-49fd-ad8b-19fee82ccac9 None admin] PATH_INFO: `/v3/users/a2489171853b43e8bc7ff724ab503ed9` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:49.104244 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fa1a7-6e1a-49fd-ad8b-19fee82ccac9 None admin] RBAC: Authorizing `identity:get_user(user_id=a2489171853b43e8bc7ff724ab503ed9)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:49.105944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fa1a7-6e1a-49fd-ad8b-19fee82ccac9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:49.107841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 105/202] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:48 2022] GET /identity/v3/users/a2489171853b43e8bc7ff724ab503ed9 => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:49.109149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fa1a7-6e1a-49fd-ad8b-19fee82ccac9 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:49.183201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0995d-207d-43ff-8b13-da0a8cb59e74 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:49.194614 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:49.200078 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:49.217257 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba69aa-237f-4113-9143-32d19e3e7372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:49.218108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba69aa-237f-4113-9143-32d19e3e7372 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:49.218394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba69aa-237f-4113-9143-32d19e3e7372 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:49.218666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba69aa-237f-4113-9143-32d19e3e7372 None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:49.219859 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba69aa-237f-4113-9143-32d19e3e7372 None admin] RBAC: Authorizing `identity:get_project(project_id=45bac146c9d842bf82503d3f2b145128)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:49.221370 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba69aa-237f-4113-9143-32d19e3e7372 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:49.221605 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fa1a7-6e1a-49fd-ad8b-19fee82ccac9 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:49.223951 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 98/203] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:49 2022] GET /identity/v3/projects/45bac146c9d842bf82503d3f2b145128 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:49.225412 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba69aa-237f-4113-9143-32d19e3e7372 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:49.260543 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ecc92-59c4-4ef9-8927-c09562aa073d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:49.261475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ecc92-59c4-4ef9-8927-c09562aa073d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:49.261911 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ecc92-59c4-4ef9-8927-c09562aa073d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:49.262389 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ecc92-59c4-4ef9-8927-c09562aa073d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:49.263563 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ecc92-59c4-4ef9-8927-c09562aa073d None admin] RBAC: Adding query filter params (role.id=27fc14631c494ca0b2dc7f708be016c2, scope.project.id=45bac146c9d842bf82503d3f2b145128, user.id=5c93b65311c34f30bf0f65e27782e21d) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:49.264021 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ecc92-59c4-4ef9-8927-c09562aa073d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:49.265687 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ecc92-59c4-4ef9-8927-c09562aa073d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:49.271859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 106/204] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:48:49 2022] GET /identity/v3/role_assignments?user.id=5c93b65311c34f30bf0f65e27782e21d&scope.project.id=45bac146c9d842bf82503d3f2b145128&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:49.272923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ecc92-59c4-4ef9-8927-c09562aa073d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:49.339198 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba69aa-237f-4113-9143-32d19e3e7372 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:49.378757 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6bf2d-36be-4518-a7fb-a14b1d89ad48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:49.379415 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6bf2d-36be-4518-a7fb-a14b1d89ad48 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:49.380946 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6bf2d-36be-4518-a7fb-a14b1d89ad48 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:49.381377 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6bf2d-36be-4518-a7fb-a14b1d89ad48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:49.449926 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ecc92-59c4-4ef9-8927-c09562aa073d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:49.498234 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c552e61-1a2a-4474-a210-d1d5e7d377a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:49.499208 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c552e61-1a2a-4474-a210-d1d5e7d377a8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:49.499637 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c552e61-1a2a-4474-a210-d1d5e7d377a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:49.500164 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c552e61-1a2a-4474-a210-d1d5e7d377a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:49.507546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c552e61-1a2a-4474-a210-d1d5e7d377a8 None admin] RBAC: Adding query filter params (role.id=394e165201dc41c8b3255de480821b1f, scope.project.id=45bac146c9d842bf82503d3f2b145128, user.id=5c93b65311c34f30bf0f65e27782e21d) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:49.507866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c552e61-1a2a-4474-a210-d1d5e7d377a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:49.509369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c552e61-1a2a-4474-a210-d1d5e7d377a8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:49.523054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 107/205] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:48:49 2022] GET /identity/v3/role_assignments?user.id=5c93b65311c34f30bf0f65e27782e21d&scope.project.id=45bac146c9d842bf82503d3f2b145128&role.id=394e165201dc41c8b3255de480821b1f => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:49.534349 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:49.545737 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6bf2d-36be-4518-a7fb-a14b1d89ad48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:49.547400 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6bf2d-36be-4518-a7fb-a14b1d89ad48 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:49.666833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 99/206] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:48:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:48:49.672376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c552e61-1a2a-4474-a210-d1d5e7d377a8 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:49.682225 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:49.791937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c552e61-1a2a-4474-a210-d1d5e7d377a8 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:49.826693 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cdc93-dba4-4792-856a-58ea917a256a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:49.827604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cdc93-dba4-4792-856a-58ea917a256a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:49.827886 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cdc93-dba4-4792-856a-58ea917a256a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:49.828159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cdc93-dba4-4792-856a-58ea917a256a None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:49.829400 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888cdc93-dba4-4792-856a-58ea917a256a None admin] RBAC: Authorizing `identity:get_project(project_id=b82ee754849c4779b408e856c429ab9b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:49.830820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888cdc93-dba4-4792-856a-58ea917a256a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:49.832624 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 108/207] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:49 2022] GET /identity/v3/projects/b82ee754849c4779b408e856c429ab9b => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:49.844506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6bf2d-36be-4518-a7fb-a14b1d89ad48 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:49.946268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:49.961625 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6bf2d-36be-4518-a7fb-a14b1d89ad48 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:49.995916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72459610-002d-4a7b-841a-b2fb6a9dcfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:49.996584 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72459610-002d-4a7b-841a-b2fb6a9dcfd7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:49.996873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72459610-002d-4a7b-841a-b2fb6a9dcfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:49.997157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72459610-002d-4a7b-841a-b2fb6a9dcfd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:49.998090 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72459610-002d-4a7b-841a-b2fb6a9dcfd7 None admin] RBAC: Adding query filter params (role.id=a2403691e5c440df989bc6aad7118272, scope.project.id=b82ee754849c4779b408e856c429ab9b, user.id=a2489171853b43e8bc7ff724ab503ed9) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:49.998406 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72459610-002d-4a7b-841a-b2fb6a9dcfd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:49.999893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72459610-002d-4a7b-841a-b2fb6a9dcfd7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:50.009099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 100/208] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:48:49 2022] GET /identity/v3/role_assignments?user.id=a2489171853b43e8bc7ff724ab503ed9&scope.project.id=b82ee754849c4779b408e856c429ab9b&role.id=a2403691e5c440df989bc6aad7118272 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:50.452547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c970445-7def-4937-bc5c-dd5214c854c3 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:50.455952 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c970445-7def-4937-bc5c-dd5214c854c3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:50.455952 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c970445-7def-4937-bc5c-dd5214c854c3 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:50.455952 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 109/209] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:50.462663 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14295c56-08d7-4b62-8143-0a906b093932 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:50.471942 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14295c56-08d7-4b62-8143-0a906b093932 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:50.471942 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14295c56-08d7-4b62-8143-0a906b093932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:50.482813 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-14295c56-08d7-4b62-8143-0a906b093932 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:50.484038 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-14295c56-08d7-4b62-8143-0a906b093932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:50.523959 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb33914-a1ed-4b7a-9d35-cbfebd2317b4 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:50.524563 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb33914-a1ed-4b7a-9d35-cbfebd2317b4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:50.525078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb33914-a1ed-4b7a-9d35-cbfebd2317b4 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:50.526914 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 110/210] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:50.532064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b94f82-5907-46d7-ae74-26afd0fd539f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:50.532587 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b94f82-5907-46d7-ae74-26afd0fd539f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:50.533058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b94f82-5907-46d7-ae74-26afd0fd539f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:50.553401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-54b94f82-5907-46d7-ae74-26afd0fd539f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:50.554939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-54b94f82-5907-46d7-ae74-26afd0fd539f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:50.568859 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 101/211] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:50.595053 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14295c56-08d7-4b62-8143-0a906b093932 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:50.643571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 111/212] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:48:50.663788 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b94f82-5907-46d7-ae74-26afd0fd539f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:50.667333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:50.703857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:50.730472 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14295c56-08d7-4b62-8143-0a906b093932 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:50.783269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b94f82-5907-46d7-ae74-26afd0fd539f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:50.783456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9bc3b-5516-46f8-9859-6bc27e778282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:50.784249 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9bc3b-5516-46f8-9859-6bc27e778282 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:50.784559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9bc3b-5516-46f8-9859-6bc27e778282 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:50.784853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9bc3b-5516-46f8-9859-6bc27e778282 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:50.793300 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9bc3b-5516-46f8-9859-6bc27e778282 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:50.793300 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9bc3b-5516-46f8-9859-6bc27e778282 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:50.793300 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 102/213] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:48:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:50.797076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9bc3b-5516-46f8-9859-6bc27e778282 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:50.829758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071631c5-982d-42e8-a479-cdea25671830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:50.830585 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071631c5-982d-42e8-a479-cdea25671830 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:50.830871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071631c5-982d-42e8-a479-cdea25671830 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:50.831148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071631c5-982d-42e8-a479-cdea25671830 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:50.843880 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071631c5-982d-42e8-a479-cdea25671830 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:50.852528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-071631c5-982d-42e8-a479-cdea25671830 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:50.852528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:50.852528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:50.852528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:50.852528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:50.852528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:50.852528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:50.852528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:50.852528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:50.852528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:50.852528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:50.852528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:50.852528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:50.852528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:50.852528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:50.852528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:50.853391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:50.853391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:50.853391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:50.853391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:50.853391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:50.853391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:50.853391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:50.853391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:50.853391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:50.853391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:50.853391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:50.853391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:50.853391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:50.853391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:50.853391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:50.853391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:50.854159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:50.854159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:50.854159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:50.854159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:50.854159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:50.854159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:50.854159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:50.854159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:50.854159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:50.854159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:50.854159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:50.854159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:50.854159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 112/214] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:48:50.859465 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071631c5-982d-42e8-a479-cdea25671830 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:50.925920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:50.929039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9bc3b-5516-46f8-9859-6bc27e778282 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:50.967458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bfb62-34c2-4f8a-974c-ecb00f45df62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:50.968056 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bfb62-34c2-4f8a-974c-ecb00f45df62 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:50.968356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bfb62-34c2-4f8a-974c-ecb00f45df62 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:50.968631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bfb62-34c2-4f8a-974c-ecb00f45df62 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:50.969357 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bfb62-34c2-4f8a-974c-ecb00f45df62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:50.971046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bfb62-34c2-4f8a-974c-ecb00f45df62 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:50.977563 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071631c5-982d-42e8-a479-cdea25671830 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:51.021463 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27b36d-6394-4dd8-85ea-445d72710fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:51.022180 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27b36d-6394-4dd8-85ea-445d72710fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:51.022464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27b36d-6394-4dd8-85ea-445d72710fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:51.022746 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27b36d-6394-4dd8-85ea-445d72710fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:51.023587 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27b36d-6394-4dd8-85ea-445d72710fa7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:51.026646 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27b36d-6394-4dd8-85ea-445d72710fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:51.026646 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27b36d-6394-4dd8-85ea-445d72710fa7 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:51.034125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 113/215] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:48:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:51.040376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27b36d-6394-4dd8-85ea-445d72710fa7 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:51.087295 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 103/216] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 14 18:48:50 2022] POST /identity/v3/users => generated 297 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:48:51.161964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27b36d-6394-4dd8-85ea-445d72710fa7 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:51.191198 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:51.201527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51df3c-bc32-4f5c-914e-b47892576174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:51.202425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51df3c-bc32-4f5c-914e-b47892576174 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:51.202715 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51df3c-bc32-4f5c-914e-b47892576174 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:51.202996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51df3c-bc32-4f5c-914e-b47892576174 None admin] PATH_INFO: `/v3/users/3abf085883f14d268b61efb2618311a5` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:51.219874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a51df3c-bc32-4f5c-914e-b47892576174 None admin] RBAC: Authorizing `identity:get_user(user_id=3abf085883f14d268b61efb2618311a5)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:51.221439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a51df3c-bc32-4f5c-914e-b47892576174 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:51.223459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 114/217] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:51 2022] GET /identity/v3/users/3abf085883f14d268b61efb2618311a5 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:51.229171 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bfb62-34c2-4f8a-974c-ecb00f45df62 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:51.233548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:51.342953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bfb62-34c2-4f8a-974c-ecb00f45df62 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:51.380729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee56cd8-5040-4b9d-b0ac-a13715fb311a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:51.381779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee56cd8-5040-4b9d-b0ac-a13715fb311a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:51.382268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee56cd8-5040-4b9d-b0ac-a13715fb311a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:51.382761 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee56cd8-5040-4b9d-b0ac-a13715fb311a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:51.458457 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee56cd8-5040-4b9d-b0ac-a13715fb311a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:51.460317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee56cd8-5040-4b9d-b0ac-a13715fb311a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:51.463061 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:51.569170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 104/218] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:48:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:48:51.575121 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51df3c-bc32-4f5c-914e-b47892576174 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:51.693465 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51df3c-bc32-4f5c-914e-b47892576174 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:51.735114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5a52e-93cf-4599-b1ec-80567eedf31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:51.736028 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5a52e-93cf-4599-b1ec-80567eedf31a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:51.736317 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5a52e-93cf-4599-b1ec-80567eedf31a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:51.736598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5a52e-93cf-4599-b1ec-80567eedf31a None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:51.737813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5a52e-93cf-4599-b1ec-80567eedf31a None admin] RBAC: Authorizing `identity:get_project(project_id=45bac146c9d842bf82503d3f2b145128)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:51.739992 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5a52e-93cf-4599-b1ec-80567eedf31a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:51.742046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 115/219] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:51 2022] GET /identity/v3/projects/45bac146c9d842bf82503d3f2b145128 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:51.747974 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee56cd8-5040-4b9d-b0ac-a13715fb311a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:51.751309 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622477a1-7084-4b93-ab0b-702470ea330e None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:51.751606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622477a1-7084-4b93-ab0b-702470ea330e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:51.751888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622477a1-7084-4b93-ab0b-702470ea330e None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:51.753234 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 116/220] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:51.759037 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d949eb3-2ac2-4705-b4c3-fd206f636eb4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:51.759359 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d949eb3-2ac2-4705-b4c3-fd206f636eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:51.759647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d949eb3-2ac2-4705-b4c3-fd206f636eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:51.779720 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-1d949eb3-2ac2-4705-b4c3-fd206f636eb4 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:51.781294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1d949eb3-2ac2-4705-b4c3-fd206f636eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:51.851538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:51.861996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 117/221] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:48:51.875572 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d75ee-a0f3-4412-a990-4fae70ca4d02 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:51.875908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d75ee-a0f3-4412-a990-4fae70ca4d02 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:51.876201 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d75ee-a0f3-4412-a990-4fae70ca4d02 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:51.877708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 118/222] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:51 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:51.884655 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0259f78-ff01-4b0b-9674-bb247d1818e3 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:51.884955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0259f78-ff01-4b0b-9674-bb247d1818e3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:51.885245 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0259f78-ff01-4b0b-9674-bb247d1818e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:51.901978 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee56cd8-5040-4b9d-b0ac-a13715fb311a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:51.904876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:51.906624 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d0259f78-ff01-4b0b-9674-bb247d1818e3 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:51.907882 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d0259f78-ff01-4b0b-9674-bb247d1818e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:51.947098 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37dbe1-25cc-4838-a684-3a17e420ceb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:51.947902 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37dbe1-25cc-4838-a684-3a17e420ceb6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:51.948413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37dbe1-25cc-4838-a684-3a17e420ceb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:51.948759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37dbe1-25cc-4838-a684-3a17e420ceb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:51.950502 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37dbe1-25cc-4838-a684-3a17e420ceb6 None admin] RBAC: Adding query filter params (role.id=ffed4493824a4040977d2cd1d057e07b, scope.project.id=45bac146c9d842bf82503d3f2b145128, user.id=3abf085883f14d268b61efb2618311a5) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:51.950892 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37dbe1-25cc-4838-a684-3a17e420ceb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:51.952466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37dbe1-25cc-4838-a684-3a17e420ceb6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:51.960044 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 105/223] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:48:51 2022] GET /identity/v3/role_assignments?user.id=3abf085883f14d268b61efb2618311a5&scope.project.id=45bac146c9d842bf82503d3f2b145128&role.id=ffed4493824a4040977d2cd1d057e07b => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:51.960771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37dbe1-25cc-4838-a684-3a17e420ceb6 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:51.993911 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 119/224] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:52.022849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0259f78-ff01-4b0b-9674-bb247d1818e3 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:52.123048 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37dbe1-25cc-4838-a684-3a17e420ceb6 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:52.168360 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d188df0-292e-49e7-8449-1dba211e090f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:52.169289 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d188df0-292e-49e7-8449-1dba211e090f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:52.169580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d188df0-292e-49e7-8449-1dba211e090f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:52.169860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d188df0-292e-49e7-8449-1dba211e090f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:52.181825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d188df0-292e-49e7-8449-1dba211e090f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:52.191642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-8d188df0-292e-49e7-8449-1dba211e090f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:48:52.191642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:52.191642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:52.191642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:52.191642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:52.191642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:52.191642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:52.191642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:52.191642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:52.191642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:52.191642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:52.191642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:52.191642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:52.191642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:52.191642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:52.191642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:52.193034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:52.193034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:52.193034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:52.193034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:52.193034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:52.193034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:52.193034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:52.193034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:52.193034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:52.193034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:52.193034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:52.193034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:52.193034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:52.193034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:52.193034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:52.193034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:52.193794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:52.193794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:52.193794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:52.193794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:52.193794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:52.193794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:52.193794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:52.193794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:52.193794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:52.193794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:52.193794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:48:52.193794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:48:52.193794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 106/225] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:48:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:52.195967 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0259f78-ff01-4b0b-9674-bb247d1818e3 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:52.200925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d188df0-292e-49e7-8449-1dba211e090f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:52.219339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:52.240081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:52.288479 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84247074-e408-418a-9f45-412ae842035f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:52.289303 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84247074-e408-418a-9f45-412ae842035f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:52.289599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84247074-e408-418a-9f45-412ae842035f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:52.289881 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84247074-e408-418a-9f45-412ae842035f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:52.319431 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d188df0-292e-49e7-8449-1dba211e090f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:52.324074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84247074-e408-418a-9f45-412ae842035f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:52.338654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-84247074-e408-418a-9f45-412ae842035f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:48:52.338654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:52.338654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:52.338654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:52.338654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:52.338654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:52.338654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:52.338654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:52.338654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:52.338654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:52.338654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:52.338654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:52.338654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:52.338654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:52.338654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:52.338654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:52.339593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:52.339593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:52.339593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:52.339593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:52.339593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:52.339593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:52.339593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:52.339593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:52.339593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:52.339593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:52.339593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:52.339593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:52.339593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:52.339593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:52.339593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:52.339593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:52.340364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:52.340364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:52.340364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:52.340364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:52.340364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:52.340364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:52.340364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:52.340364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:52.340364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:52.340364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:52.340364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:48:52.340364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:52.340963 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 120/226] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:48:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:48:52.345878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84247074-e408-418a-9f45-412ae842035f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:52.359064 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e99c92-fbfd-449c-a926-d96901485a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:52.360127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e99c92-fbfd-449c-a926-d96901485a4a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:52.360651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e99c92-fbfd-449c-a926-d96901485a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:52.361126 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e99c92-fbfd-449c-a926-d96901485a4a None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:52.362167 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e99c92-fbfd-449c-a926-d96901485a4a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:52.362688 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e99c92-fbfd-449c-a926-d96901485a4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:52.364498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e99c92-fbfd-449c-a926-d96901485a4a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:52.373815 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 107/227] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:48:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:52.384336 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e99c92-fbfd-449c-a926-d96901485a4a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:52.519519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:52.533639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e99c92-fbfd-449c-a926-d96901485a4a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:52.571724 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84247074-e408-418a-9f45-412ae842035f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:52.610475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddea74-4c75-4649-a0af-1190475ce517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:52.611249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddea74-4c75-4649-a0af-1190475ce517 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:52.611462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddea74-4c75-4649-a0af-1190475ce517 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:52.611743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddea74-4c75-4649-a0af-1190475ce517 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:52.612493 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ddea74-4c75-4649-a0af-1190475ce517 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:52.612806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ddea74-4c75-4649-a0af-1190475ce517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:52.614315 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ddea74-4c75-4649-a0af-1190475ce517 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:52.616749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487cbe4-f1dd-4475-946f-3bc25c87d4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:52.617638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487cbe4-f1dd-4475-946f-3bc25c87d4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:52.617924 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487cbe4-f1dd-4475-946f-3bc25c87d4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:52.618199 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487cbe4-f1dd-4475-946f-3bc25c87d4d9 None admin] PATH_INFO: `/v3/users/5c93b65311c34f30bf0f65e27782e21d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:52.623932 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 121/228] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:52.624068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487cbe4-f1dd-4475-946f-3bc25c87d4d9 None admin] RBAC: Authorizing `identity:get_user(user_id=5c93b65311c34f30bf0f65e27782e21d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:52.625344 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487cbe4-f1dd-4475-946f-3bc25c87d4d9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:52.631283 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 108/229] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:52 2022] GET /identity/v3/users/5c93b65311c34f30bf0f65e27782e21d => generated 283 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:52.632380 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddea74-4c75-4649-a0af-1190475ce517 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:52.633978 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487cbe4-f1dd-4475-946f-3bc25c87d4d9 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:52.671297 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:52.803980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddea74-4c75-4649-a0af-1190475ce517 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:52.847227 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487cbe4-f1dd-4475-946f-3bc25c87d4d9 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:52.888103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c34703-850a-4788-803c-067db5b096ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:52.889254 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c34703-850a-4788-803c-067db5b096ab None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:52.889721 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c34703-850a-4788-803c-067db5b096ab None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:52.890175 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c34703-850a-4788-803c-067db5b096ab None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:52.891739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c34703-850a-4788-803c-067db5b096ab None admin] RBAC: Authorizing `identity:get_project(project_id=45bac146c9d842bf82503d3f2b145128)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:52.893416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c34703-850a-4788-803c-067db5b096ab None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:52.900488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f11658-d061-42b6-a5ff-8234854e118b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:52.900488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f11658-d061-42b6-a5ff-8234854e118b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:52.900488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f11658-d061-42b6-a5ff-8234854e118b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:52.900488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f11658-d061-42b6-a5ff-8234854e118b None admin] PATH_INFO: `/v3/users/5c93b65311c34f30bf0f65e27782e21d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:52.900828 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f11658-d061-42b6-a5ff-8234854e118b None admin] RBAC: Authorizing `identity:get_user(user_id=5c93b65311c34f30bf0f65e27782e21d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:52.900828 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f11658-d061-42b6-a5ff-8234854e118b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:52.901330 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 109/230] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:52 2022] GET /identity/v3/projects/45bac146c9d842bf82503d3f2b145128 => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:52.902575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c34703-850a-4788-803c-067db5b096ab None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:52.902750 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 122/231] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:52 2022] GET /identity/v3/users/5c93b65311c34f30bf0f65e27782e21d => generated 283 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:52.907733 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f11658-d061-42b6-a5ff-8234854e118b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:52.931687 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:53.007413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c34703-850a-4788-803c-067db5b096ab None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:53.042775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da2cc2-ff5e-4665-8edc-86b3a612bf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:53.043940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da2cc2-ff5e-4665-8edc-86b3a612bf24 None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:53.044381 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da2cc2-ff5e-4665-8edc-86b3a612bf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:53.044801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da2cc2-ff5e-4665-8edc-86b3a612bf24 None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128/users/5c93b65311c34f30bf0f65e27782e21d/roles/394e165201dc41c8b3255de480821b1f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:53.050865 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da2cc2-ff5e-4665-8edc-86b3a612bf24 None admin] RBAC: Authorizing `identity:create_grant(project_id=45bac146c9d842bf82503d3f2b145128, user_id=5c93b65311c34f30bf0f65e27782e21d, role_id=394e165201dc41c8b3255de480821b1f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:53.052620 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da2cc2-ff5e-4665-8edc-86b3a612bf24 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:53.103089 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f11658-d061-42b6-a5ff-8234854e118b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:53.104941 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:53.139874 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 110/232] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:48:52 2022] PUT /identity/v3/projects/45bac146c9d842bf82503d3f2b145128/users/5c93b65311c34f30bf0f65e27782e21d/roles/394e165201dc41c8b3255de480821b1f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:48:53.165220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db983a63-a25d-404e-8da2-a1f71bd77383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:53.166522 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db983a63-a25d-404e-8da2-a1f71bd77383 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:53.167071 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db983a63-a25d-404e-8da2-a1f71bd77383 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:53.167642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db983a63-a25d-404e-8da2-a1f71bd77383 None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:53.172134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db983a63-a25d-404e-8da2-a1f71bd77383 None admin] RBAC: Authorizing `identity:get_project(project_id=45bac146c9d842bf82503d3f2b145128)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:53.174117 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db983a63-a25d-404e-8da2-a1f71bd77383 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:53.177217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 123/233] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:52 2022] GET /identity/v3/projects/45bac146c9d842bf82503d3f2b145128 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:53.182718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da2cc2-ff5e-4665-8edc-86b3a612bf24 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:53.304906 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:53.322685 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da2cc2-ff5e-4665-8edc-86b3a612bf24 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:53.371579 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d236cb-8b0c-47e4-b6a7-1cf0979c4f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:53.372550 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d236cb-8b0c-47e4-b6a7-1cf0979c4f4b None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:53.372857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d236cb-8b0c-47e4-b6a7-1cf0979c4f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:53.373135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d236cb-8b0c-47e4-b6a7-1cf0979c4f4b None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128/users/5c93b65311c34f30bf0f65e27782e21d/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:53.375193 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d236cb-8b0c-47e4-b6a7-1cf0979c4f4b None admin] RBAC: Authorizing `identity:create_grant(project_id=45bac146c9d842bf82503d3f2b145128, user_id=5c93b65311c34f30bf0f65e27782e21d, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:53.376755 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d236cb-8b0c-47e4-b6a7-1cf0979c4f4b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:53.507083 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 111/234] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:48:53 2022] PUT /identity/v3/projects/45bac146c9d842bf82503d3f2b145128/users/5c93b65311c34f30bf0f65e27782e21d/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 18:48:53.915972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0f3a0-eab9-4f30-bbf6-9c06f07f141e None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:53.916476 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0f3a0-eab9-4f30-bbf6-9c06f07f141e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:53.916549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0f3a0-eab9-4f30-bbf6-9c06f07f141e None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:53.918048 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 124/235] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:53.923463 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaff1ed-c33b-466b-9f56-17a2f6c35116 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:53.923902 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaff1ed-c33b-466b-9f56-17a2f6c35116 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:53.924079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaff1ed-c33b-466b-9f56-17a2f6c35116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:53.932131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd0cd3-145b-4fd4-992d-322d3ced16df None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:53.932423 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd0cd3-145b-4fd4-992d-322d3ced16df None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:53.932712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd0cd3-145b-4fd4-992d-322d3ced16df None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:53.934206 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 125/236] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:53.956414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-baaff1ed-c33b-466b-9f56-17a2f6c35116 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:53.957073 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39bdc1-3493-47c7-b0f1-b776c920ad62 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:53.957890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39bdc1-3493-47c7-b0f1-b776c920ad62 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:53.958188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39bdc1-3493-47c7-b0f1-b776c920ad62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:53.958288 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-baaff1ed-c33b-466b-9f56-17a2f6c35116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:53.984856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5f39bdc1-3493-47c7-b0f1-b776c920ad62 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:53.986052 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5f39bdc1-3493-47c7-b0f1-b776c920ad62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:54.068613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 112/237] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:48:54.097026 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 126/238] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:54.104534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaff1ed-c33b-466b-9f56-17a2f6c35116 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:54.126446 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f39bdc1-3493-47c7-b0f1-b776c920ad62 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:54.245479 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:54.271430 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaff1ed-c33b-466b-9f56-17a2f6c35116 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:54.279984 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:54.283892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f39bdc1-3493-47c7-b0f1-b776c920ad62 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:54.318384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa0984-a16b-405e-884d-d78e47d6c512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:54.319868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa0984-a16b-405e-884d-d78e47d6c512 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:54.320164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa0984-a16b-405e-884d-d78e47d6c512 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:54.320776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa0984-a16b-405e-884d-d78e47d6c512 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:54.328959 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3067ef-02a3-4ebd-a819-1103b62c2e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:54.329916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3067ef-02a3-4ebd-a819-1103b62c2e81 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:54.330300 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3067ef-02a3-4ebd-a819-1103b62c2e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:54.330580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3067ef-02a3-4ebd-a819-1103b62c2e81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:54.331790 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3067ef-02a3-4ebd-a819-1103b62c2e81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:54.333742 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa0984-a16b-405e-884d-d78e47d6c512 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:54.333925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3067ef-02a3-4ebd-a819-1103b62c2e81 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:54.335763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 127/239] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:48:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:54.337047 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b643e6-8413-4217-af30-531f96273dd6 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:54.337340 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b643e6-8413-4217-af30-531f96273dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:54.337619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b643e6-8413-4217-af30-531f96273dd6 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:54.338861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 128/240] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:54.342241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-acaa0984-a16b-405e-884d-d78e47d6c512 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 18:48:54.342241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:54.342241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:54.342241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:54.342241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:54.342241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:54.342241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:54.342241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:54.342241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:54.342241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:54.342241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:54.342241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:54.342241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:54.342241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:54.342241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:54.342241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:54.343565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:54.343565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:54.343565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:54.343565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:54.343565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:54.343565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:54.343565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:54.343565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:54.343565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:54.343565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:54.343565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:54.343565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:54.343565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:54.343565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:54.343565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:54.343565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:54.345376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:54.345376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:54.345376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:54.345376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:54.345376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:54.345376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:54.345376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:54.345376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:54.345376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:54.345376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:54.345376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 18:48:54.345376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:48:54.345376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 113/241] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:54.346895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b643e6-8413-4217-af30-531f96273dd6 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:54.351201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9377a33-5133-4e5c-a580-dc937d3afc05 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:54.351201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9377a33-5133-4e5c-a580-dc937d3afc05 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:54.351201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9377a33-5133-4e5c-a580-dc937d3afc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:54.371926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d9377a33-5133-4e5c-a580-dc937d3afc05 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:54.373703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d9377a33-5133-4e5c-a580-dc937d3afc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:54.462996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b643e6-8413-4217-af30-531f96273dd6 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:54.470030 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 114/242] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:54.470919 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9377a33-5133-4e5c-a580-dc937d3afc05 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:54.521299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765653a6-257d-47c6-ab76-c93523d9ccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:54.521971 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765653a6-257d-47c6-ab76-c93523d9ccb7 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:54.522258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765653a6-257d-47c6-ab76-c93523d9ccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:54.522581 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765653a6-257d-47c6-ab76-c93523d9ccb7 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:54.523329 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765653a6-257d-47c6-ab76-c93523d9ccb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:54.524853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765653a6-257d-47c6-ab76-c93523d9ccb7 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:54.550159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:54.584621 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 129/243] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 14 18:48:54 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:48:54.585468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765653a6-257d-47c6-ab76-c93523d9ccb7 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:54.586912 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9377a33-5133-4e5c-a580-dc937d3afc05 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:54.619426 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:54.659794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69105f9d-b268-41eb-a275-450c6a39e1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:54.660529 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69105f9d-b268-41eb-a275-450c6a39e1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:54.660924 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69105f9d-b268-41eb-a275-450c6a39e1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:54.661267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69105f9d-b268-41eb-a275-450c6a39e1e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:54.662060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69105f9d-b268-41eb-a275-450c6a39e1e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:54.662377 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69105f9d-b268-41eb-a275-450c6a39e1e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:54.663999 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69105f9d-b268-41eb-a275-450c6a39e1e4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:54.685196 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 115/244] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:48:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:54.691213 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69105f9d-b268-41eb-a275-450c6a39e1e4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:54.713847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765653a6-257d-47c6-ab76-c93523d9ccb7 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:54.759833 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0c068-fd9d-4d32-b943-9f379e988187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:54.760684 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0c068-fd9d-4d32-b943-9f379e988187 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:54.760989 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0c068-fd9d-4d32-b943-9f379e988187 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:54.761274 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0c068-fd9d-4d32-b943-9f379e988187 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:54.776785 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0c068-fd9d-4d32-b943-9f379e988187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:54.787817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-95f0c068-fd9d-4d32-b943-9f379e988187 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:54.787817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:54.787817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:54.787817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:54.787817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:54.787817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:54.787817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:54.787817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:54.787817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:54.787817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:54.787817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:54.787817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:54.787817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:54.787817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:54.787817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:54.787817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:54.789830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:54.789830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:54.789830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:54.789830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:54.789830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:54.789830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:54.789830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:54.789830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:54.789830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:54.789830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:54.789830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:54.789830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:54.789830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:54.789830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:54.789830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:54.789830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:54.791154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:54.791154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:54.791154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:54.791154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:54.791154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:54.791154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:54.791154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:54.791154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:54.791154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:54.791154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:54.791154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:54.791154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:54.791154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 130/245] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:54.795195 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0c068-fd9d-4d32-b943-9f379e988187 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:54.830788 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69105f9d-b268-41eb-a275-450c6a39e1e4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:54.860588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:54.869969 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9f555-763a-40c1-907d-6f2f0ec11e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:54.870962 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9f555-763a-40c1-907d-6f2f0ec11e6b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:54.871284 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9f555-763a-40c1-907d-6f2f0ec11e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:54.871717 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9f555-763a-40c1-907d-6f2f0ec11e6b None admin] PATH_INFO: `/v3/users/22d68157f8464127a6a3a26814cb4f36` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:54.888392 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd9f555-763a-40c1-907d-6f2f0ec11e6b None admin] RBAC: Authorizing `identity:get_user(user_id=22d68157f8464127a6a3a26814cb4f36)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:54.889320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:54.890512 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd9f555-763a-40c1-907d-6f2f0ec11e6b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:54.892430 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 116/246] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:54 2022] GET /identity/v3/users/22d68157f8464127a6a3a26814cb4f36 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:54.897922 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9f555-763a-40c1-907d-6f2f0ec11e6b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:54.907696 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0c068-fd9d-4d32-b943-9f379e988187 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:54.943442 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001797c3-d79f-475c-a901-7259bf3a9bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:54.944418 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001797c3-d79f-475c-a901-7259bf3a9bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:54.944418 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001797c3-d79f-475c-a901-7259bf3a9bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:54.944704 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001797c3-d79f-475c-a901-7259bf3a9bf2 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:54.945426 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001797c3-d79f-475c-a901-7259bf3a9bf2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:54.945786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001797c3-d79f-475c-a901-7259bf3a9bf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:54.947265 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001797c3-d79f-475c-a901-7259bf3a9bf2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:54.956799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 131/247] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:48:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:54.963449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001797c3-d79f-475c-a901-7259bf3a9bf2 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:55.018308 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9f555-763a-40c1-907d-6f2f0ec11e6b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:55.058520 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd9323-abec-4d8e-8985-8ac19fffc722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:55.059117 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd9323-abec-4d8e-8985-8ac19fffc722 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:55.059442 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd9323-abec-4d8e-8985-8ac19fffc722 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:55.059725 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd9323-abec-4d8e-8985-8ac19fffc722 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:55.082384 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001797c3-d79f-475c-a901-7259bf3a9bf2 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:55.118882 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f21eb-2fcc-49ad-8710-ce0b63ac39dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:55.120153 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f21eb-2fcc-49ad-8710-ce0b63ac39dc None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:55.120643 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f21eb-2fcc-49ad-8710-ce0b63ac39dc None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:55.121119 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f21eb-2fcc-49ad-8710-ce0b63ac39dc None admin] PATH_INFO: `/v3/users/3abf085883f14d268b61efb2618311a5` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:55.122558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f21eb-2fcc-49ad-8710-ce0b63ac39dc None admin] RBAC: Authorizing `identity:get_user(user_id=3abf085883f14d268b61efb2618311a5)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:55.124283 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f21eb-2fcc-49ad-8710-ce0b63ac39dc None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:55.126175 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 132/248] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:54 2022] GET /identity/v3/users/3abf085883f14d268b61efb2618311a5 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:55.130903 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:55.132028 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f21eb-2fcc-49ad-8710-ce0b63ac39dc None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:55.143831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd9323-abec-4d8e-8985-8ac19fffc722 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:55.145565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd9323-abec-4d8e-8985-8ac19fffc722 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:55.149264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:55.247394 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f21eb-2fcc-49ad-8710-ce0b63ac39dc None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:55.272298 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 117/249] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:48:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 18:48:55.278966 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd9323-abec-4d8e-8985-8ac19fffc722 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:55.298091 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef48a9-d3cc-4940-970e-ad4cefef245b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:55.299134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef48a9-d3cc-4940-970e-ad4cefef245b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:55.299470 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef48a9-d3cc-4940-970e-ad4cefef245b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:55.299753 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef48a9-d3cc-4940-970e-ad4cefef245b None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:55.301237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef48a9-d3cc-4940-970e-ad4cefef245b None admin] RBAC: Authorizing `identity:get_project(project_id=45bac146c9d842bf82503d3f2b145128)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:55.302795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef48a9-d3cc-4940-970e-ad4cefef245b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:55.305338 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 133/250] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:55 2022] GET /identity/v3/projects/45bac146c9d842bf82503d3f2b145128 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:55.312817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef48a9-d3cc-4940-970e-ad4cefef245b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:55.404349 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:55.424191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd9323-abec-4d8e-8985-8ac19fffc722 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:55.430521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:55.466682 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef48a9-d3cc-4940-970e-ad4cefef245b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:55.491203 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe5e9a-18db-4c32-8514-81bee833023a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:55.492482 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe5e9a-18db-4c32-8514-81bee833023a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:55.492765 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe5e9a-18db-4c32-8514-81bee833023a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:55.493041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe5e9a-18db-4c32-8514-81bee833023a None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:55.494256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe5e9a-18db-4c32-8514-81bee833023a None admin] RBAC: Authorizing `identity:get_project(project_id=45bac146c9d842bf82503d3f2b145128)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:55.495757 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe5e9a-18db-4c32-8514-81bee833023a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:55.497876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 118/251] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:55 2022] GET /identity/v3/projects/45bac146c9d842bf82503d3f2b145128 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:55.498980 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661aaf3-f01e-4d0a-b4ca-2a9b4fe1ccf3 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:55.499292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661aaf3-f01e-4d0a-b4ca-2a9b4fe1ccf3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:55.499579 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661aaf3-f01e-4d0a-b4ca-2a9b4fe1ccf3 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:55.500827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 119/252] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:55.507069 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661aaf3-f01e-4d0a-b4ca-2a9b4fe1ccf3 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:55.525558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553afb1-b865-40fa-aa66-dc8d95b857ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:55.526728 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553afb1-b865-40fa-aa66-dc8d95b857ba None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:55.527122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553afb1-b865-40fa-aa66-dc8d95b857ba None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:55.527422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553afb1-b865-40fa-aa66-dc8d95b857ba None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128/users/3abf085883f14d268b61efb2618311a5/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:55.529526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553afb1-b865-40fa-aa66-dc8d95b857ba None admin] RBAC: Authorizing `identity:create_grant(project_id=45bac146c9d842bf82503d3f2b145128, user_id=3abf085883f14d268b61efb2618311a5, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:55.531125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553afb1-b865-40fa-aa66-dc8d95b857ba None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:55.588269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 134/253] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:48:55 2022] PUT /identity/v3/projects/45bac146c9d842bf82503d3f2b145128/users/3abf085883f14d268b61efb2618311a5/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:48:55.589974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3887930-f3c6-46f7-bc19-7e6c26298ade None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:55.590533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3887930-f3c6-46f7-bc19-7e6c26298ade None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:55.590968 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3887930-f3c6-46f7-bc19-7e6c26298ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:55.619096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c3887930-f3c6-46f7-bc19-7e6c26298ade None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:55.620853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c3887930-f3c6-46f7-bc19-7e6c26298ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:55.700194 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 135/254] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:55.727196 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3887930-f3c6-46f7-bc19-7e6c26298ade None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:55.729904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661aaf3-f01e-4d0a-b4ca-2a9b4fe1ccf3 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:55.775728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea10790-ec9b-4c56-9e87-55a7a610667d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:55.777192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea10790-ec9b-4c56-9e87-55a7a610667d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:55.777705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea10790-ec9b-4c56-9e87-55a7a610667d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:55.778216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea10790-ec9b-4c56-9e87-55a7a610667d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:55.779333 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea10790-ec9b-4c56-9e87-55a7a610667d None admin] RBAC: Adding query filter params (role.id=a2403691e5c440df989bc6aad7118272, scope.project.id=45bac146c9d842bf82503d3f2b145128, user.id=22d68157f8464127a6a3a26814cb4f36) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:55.779879 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea10790-ec9b-4c56-9e87-55a7a610667d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:55.781613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea10790-ec9b-4c56-9e87-55a7a610667d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:55.789194 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 120/255] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:48:55 2022] GET /identity/v3/role_assignments?user.id=22d68157f8464127a6a3a26814cb4f36&scope.project.id=45bac146c9d842bf82503d3f2b145128&role.id=a2403691e5c440df989bc6aad7118272 => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:55.816601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:55.853534 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3887930-f3c6-46f7-bc19-7e6c26298ade None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:55.894139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024d7c5-88b9-4a6c-be52-2c576cf89532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:55.895654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024d7c5-88b9-4a6c-be52-2c576cf89532 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:55.895957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024d7c5-88b9-4a6c-be52-2c576cf89532 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:55.896237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024d7c5-88b9-4a6c-be52-2c576cf89532 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:55.909145 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024d7c5-88b9-4a6c-be52-2c576cf89532 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:55.918184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-0024d7c5-88b9-4a6c-be52-2c576cf89532 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:48:55.918184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:55.918184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:55.918184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:55.918184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:55.918184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:55.918184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:55.918184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:55.918184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:55.918184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:55.918184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:55.918184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:55.918184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:55.918184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:55.918184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:55.918184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:55.919376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:55.919376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:55.919376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:55.919376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:55.919376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:55.919376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:55.919376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:55.919376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:55.919376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:55.919376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:55.919376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:55.919376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:55.919376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:55.919376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:55.919376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:55.919376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:55.920357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:55.920357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:55.920357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:55.920357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:55.920357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:55.920357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:55.920357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:55.920357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:55.920357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:55.920357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:55.920357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:48:55.920357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:55.923658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 136/256] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:48:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:48:55.929777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea10790-ec9b-4c56-9e87-55a7a610667d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:55.968607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:56.089938 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea10790-ec9b-4c56-9e87-55a7a610667d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:56.110013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb77c27-fa2a-4d3a-9d49-0c24bbd93d00 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:56.110490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb77c27-fa2a-4d3a-9d49-0c24bbd93d00 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:56.110490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb77c27-fa2a-4d3a-9d49-0c24bbd93d00 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:56.112189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 137/257] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:56.117494 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbaadad-0bad-4ef3-be8b-5bc9a693d6b1 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:56.117791 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbaadad-0bad-4ef3-be8b-5bc9a693d6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:56.118078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbaadad-0bad-4ef3-be8b-5bc9a693d6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:56.135776 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-dbbaadad-0bad-4ef3-be8b-5bc9a693d6b1 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:56.136996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-dbbaadad-0bad-4ef3-be8b-5bc9a693d6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:56.145991 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea643ce9-13ba-4621-b197-c829de5f00c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:56.146686 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea643ce9-13ba-4621-b197-c829de5f00c1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:56.146977 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea643ce9-13ba-4621-b197-c829de5f00c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:56.147286 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea643ce9-13ba-4621-b197-c829de5f00c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:56.148157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea643ce9-13ba-4621-b197-c829de5f00c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:56.148476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea643ce9-13ba-4621-b197-c829de5f00c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:56.149993 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea643ce9-13ba-4621-b197-c829de5f00c1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:56.178388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 121/258] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:48:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:56.184829 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea643ce9-13ba-4621-b197-c829de5f00c1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:56.209938 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 138/259] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:48:56.232529 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbaadad-0bad-4ef3-be8b-5bc9a693d6b1 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:56.319299 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:56.319493 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:56.343072 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea643ce9-13ba-4621-b197-c829de5f00c1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:56.363195 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbaadad-0bad-4ef3-be8b-5bc9a693d6b1 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:56.400829 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd91208-fd0f-496c-a3f9-6e1f4bb274ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:56.402335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd91208-fd0f-496c-a3f9-6e1f4bb274ae None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:56.402843 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd91208-fd0f-496c-a3f9-6e1f4bb274ae None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:56.403347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd91208-fd0f-496c-a3f9-6e1f4bb274ae None admin] PATH_INFO: `/v3/users/5c93b65311c34f30bf0f65e27782e21d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:56.404862 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd91208-fd0f-496c-a3f9-6e1f4bb274ae None admin] RBAC: Authorizing `identity:get_user(user_id=5c93b65311c34f30bf0f65e27782e21d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:56.406544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd91208-fd0f-496c-a3f9-6e1f4bb274ae None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:56.408448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 122/260] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:56 2022] GET /identity/v3/users/5c93b65311c34f30bf0f65e27782e21d => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:56.414209 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd91208-fd0f-496c-a3f9-6e1f4bb274ae None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:56.431529 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347cfe53-e1ab-4281-8b41-09a9c1e10c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:56.431529 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cfe53-e1ab-4281-8b41-09a9c1e10c75 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:56.431529 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cfe53-e1ab-4281-8b41-09a9c1e10c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:56.431529 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cfe53-e1ab-4281-8b41-09a9c1e10c75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:56.456284 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347cfe53-e1ab-4281-8b41-09a9c1e10c75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:56.468238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-347cfe53-e1ab-4281-8b41-09a9c1e10c75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:48:56.468238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:56.468238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:56.468238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:56.468238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:56.468238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:56.468238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:56.468238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:56.468238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:56.468238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:56.468238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:56.468238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:56.468238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:56.468238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:56.468238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:56.468238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:56.469122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:56.469122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:56.469122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:56.469122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:56.469122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:56.469122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:56.469122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:56.469122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:56.469122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:56.469122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:56.469122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:56.469122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:56.469122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:56.469122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:56.469122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:56.469122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:56.469893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:56.469893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:56.469893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:56.469893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:56.469893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:56.469893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:56.469893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:56.469893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:56.469893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:56.469893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:56.469893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:48:56.469893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:56.469893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 139/261] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:48:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:48:56.475570 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347cfe53-e1ab-4281-8b41-09a9c1e10c75 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:56.621991 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd91208-fd0f-496c-a3f9-6e1f4bb274ae None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:56.685837 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347cfe53-e1ab-4281-8b41-09a9c1e10c75 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:56.690228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6531eb-7cae-49bf-9009-1ac037489df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:56.690894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6531eb-7cae-49bf-9009-1ac037489df3 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:56.691191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6531eb-7cae-49bf-9009-1ac037489df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:56.691592 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6531eb-7cae-49bf-9009-1ac037489df3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:56.727531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:56.736791 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:56.767343 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2783f94-0f7b-415e-a98f-476a276fd34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:56.768130 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2783f94-0f7b-415e-a98f-476a276fd34d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:56.768492 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2783f94-0f7b-415e-a98f-476a276fd34d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:56.768847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2783f94-0f7b-415e-a98f-476a276fd34d None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:56.775185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2783f94-0f7b-415e-a98f-476a276fd34d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:56.775185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2783f94-0f7b-415e-a98f-476a276fd34d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:56.775185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2783f94-0f7b-415e-a98f-476a276fd34d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:56.790222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 140/262] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:56.795984 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2783f94-0f7b-415e-a98f-476a276fd34d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:56.848890 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6531eb-7cae-49bf-9009-1ac037489df3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:56.851005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6531eb-7cae-49bf-9009-1ac037489df3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:57.023534 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2783f94-0f7b-415e-a98f-476a276fd34d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:57.058032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 123/263] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:48:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:48:57.063917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6531eb-7cae-49bf-9009-1ac037489df3 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:57.099097 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c525898-5a14-4dd0-9c29-04290e5e6fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:57.102576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c525898-5a14-4dd0-9c29-04290e5e6fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:57.102918 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c525898-5a14-4dd0-9c29-04290e5e6fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:57.103222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c525898-5a14-4dd0-9c29-04290e5e6fc2 None admin] PATH_INFO: `/v3/users/5c93b65311c34f30bf0f65e27782e21d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:57.104456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c525898-5a14-4dd0-9c29-04290e5e6fc2 None admin] RBAC: Authorizing `identity:get_user(user_id=5c93b65311c34f30bf0f65e27782e21d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:57.105932 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c525898-5a14-4dd0-9c29-04290e5e6fc2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:57.111418 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 141/264] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:56 2022] GET /identity/v3/users/5c93b65311c34f30bf0f65e27782e21d => generated 283 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:57.116897 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c525898-5a14-4dd0-9c29-04290e5e6fc2 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:57.169045 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:57.208262 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:57.293399 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6531eb-7cae-49bf-9009-1ac037489df3 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:57.294242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c525898-5a14-4dd0-9c29-04290e5e6fc2 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:57.330863 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832137c9-e172-4a4f-a7f3-e7ade11c25ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:57.331546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832137c9-e172-4a4f-a7f3-e7ade11c25ce None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:57.331833 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832137c9-e172-4a4f-a7f3-e7ade11c25ce None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:57.332109 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832137c9-e172-4a4f-a7f3-e7ade11c25ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:57.387889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e1d47-35ee-4881-ae12-f9c53323e13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:57.388784 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e1d47-35ee-4881-ae12-f9c53323e13b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:57.389075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e1d47-35ee-4881-ae12-f9c53323e13b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:57.389358 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e1d47-35ee-4881-ae12-f9c53323e13b None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:57.390614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e1d47-35ee-4881-ae12-f9c53323e13b None admin] RBAC: Authorizing `identity:get_project(project_id=45bac146c9d842bf82503d3f2b145128)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:57.392113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e1d47-35ee-4881-ae12-f9c53323e13b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:57.394147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 124/265] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:57 2022] GET /identity/v3/projects/45bac146c9d842bf82503d3f2b145128 => generated 302 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:57.400330 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e1d47-35ee-4881-ae12-f9c53323e13b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:57.403629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832137c9-e172-4a4f-a7f3-e7ade11c25ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:57.405103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832137c9-e172-4a4f-a7f3-e7ade11c25ce None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:57.461063 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:57.513820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 142/266] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:48:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 18:48:57.521493 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832137c9-e172-4a4f-a7f3-e7ade11c25ce None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:57.607430 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e1d47-35ee-4881-ae12-f9c53323e13b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:57.633462 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:57.671941 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302974aa-bd54-4840-adcb-67adbad7ed36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:57.672605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302974aa-bd54-4840-adcb-67adbad7ed36 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:57.672896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302974aa-bd54-4840-adcb-67adbad7ed36 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:57.673180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302974aa-bd54-4840-adcb-67adbad7ed36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:57.674022 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302974aa-bd54-4840-adcb-67adbad7ed36 None admin] RBAC: Adding query filter params (role.id=394e165201dc41c8b3255de480821b1f, scope.project.id=45bac146c9d842bf82503d3f2b145128, user.id=5c93b65311c34f30bf0f65e27782e21d) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:57.674374 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302974aa-bd54-4840-adcb-67adbad7ed36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:57.675872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302974aa-bd54-4840-adcb-67adbad7ed36 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:57.688755 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 125/267] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:48:57 2022] GET /identity/v3/role_assignments?user.id=5c93b65311c34f30bf0f65e27782e21d&scope.project.id=45bac146c9d842bf82503d3f2b145128&role.id=394e165201dc41c8b3255de480821b1f => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:57.693987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832137c9-e172-4a4f-a7f3-e7ade11c25ce None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:57.732312 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8596eee-adbb-4c64-bc7c-12207ae5b347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:57.733190 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8596eee-adbb-4c64-bc7c-12207ae5b347 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:57.733495 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8596eee-adbb-4c64-bc7c-12207ae5b347 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:57.733760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8596eee-adbb-4c64-bc7c-12207ae5b347 None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:57.734999 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8596eee-adbb-4c64-bc7c-12207ae5b347 None admin] RBAC: Authorizing `identity:get_project(project_id=45bac146c9d842bf82503d3f2b145128)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:57.736560 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8596eee-adbb-4c64-bc7c-12207ae5b347 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:57.739280 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 143/268] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:57 2022] GET /identity/v3/projects/45bac146c9d842bf82503d3f2b145128 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:57.745230 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302974aa-bd54-4840-adcb-67adbad7ed36 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:57.854160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302974aa-bd54-4840-adcb-67adbad7ed36 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:57.889953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf5d01-f2e8-4d56-bfa7-3dc5d5e5340d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:57.890550 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf5d01-f2e8-4d56-bfa7-3dc5d5e5340d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:57.890844 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf5d01-f2e8-4d56-bfa7-3dc5d5e5340d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:57.891129 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf5d01-f2e8-4d56-bfa7-3dc5d5e5340d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:57.892001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf5d01-f2e8-4d56-bfa7-3dc5d5e5340d None admin] RBAC: Adding query filter params (role.id=27fc14631c494ca0b2dc7f708be016c2, scope.project.id=45bac146c9d842bf82503d3f2b145128, user.id=5c93b65311c34f30bf0f65e27782e21d) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:57.892324 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf5d01-f2e8-4d56-bfa7-3dc5d5e5340d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:57.893796 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf5d01-f2e8-4d56-bfa7-3dc5d5e5340d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:57.900897 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 126/269] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:48:57 2022] GET /identity/v3/role_assignments?user.id=5c93b65311c34f30bf0f65e27782e21d&scope.project.id=45bac146c9d842bf82503d3f2b145128&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:58.080971 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa4f0a-b249-4565-8b0d-91a408f19587 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:58.081320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa4f0a-b249-4565-8b0d-91a408f19587 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:58.081611 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa4f0a-b249-4565-8b0d-91a408f19587 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:58.083917 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 144/270] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:58.100831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba18fea-88ef-47f4-95a8-48cdc363aaed None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:58.101473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba18fea-88ef-47f4-95a8-48cdc363aaed None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:58.102055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba18fea-88ef-47f4-95a8-48cdc363aaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:58.127086 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-2ba18fea-88ef-47f4-95a8-48cdc363aaed None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:58.128414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2ba18fea-88ef-47f4-95a8-48cdc363aaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:58.176944 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:58.204996 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 127/271] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:58.235888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa4f0a-b249-4565-8b0d-91a408f19587 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:58.262011 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:58.355057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa4f0a-b249-4565-8b0d-91a408f19587 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:58.391611 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49906f33-6e31-49ab-b8e5-8e48feb13f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:58.392768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49906f33-6e31-49ab-b8e5-8e48feb13f0e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:58.393251 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49906f33-6e31-49ab-b8e5-8e48feb13f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:58.393720 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49906f33-6e31-49ab-b8e5-8e48feb13f0e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:58.405945 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49906f33-6e31-49ab-b8e5-8e48feb13f0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:58.414573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-49906f33-6e31-49ab-b8e5-8e48feb13f0e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:58.414573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:58.414573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:58.414573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:58.414573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:58.414573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:58.414573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:58.414573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:58.414573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:58.414573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:58.414573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:58.414573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:58.414573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:58.414573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:58.414573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:58.414573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:58.415454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:58.415454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:58.415454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:58.415454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:58.415454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:58.415454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:58.415454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:58.415454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:58.415454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:58.415454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:58.415454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:58.415454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:58.415454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:58.415454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:58.415454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:58.415454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:58.416420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:58.416420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:58.416420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:58.416420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:58.416420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:58.416420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:58.416420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:58.416420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:58.416420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:58.416420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:58.416420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:58.416420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:58.419082 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 145/272] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:58.425069 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba18fea-88ef-47f4-95a8-48cdc363aaed None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:58.528344 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6379b4-9e4e-4814-adb0-24fb1a337809 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:58.528997 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6379b4-9e4e-4814-adb0-24fb1a337809 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:58.529497 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6379b4-9e4e-4814-adb0-24fb1a337809 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:58.531327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 146/273] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:58.538948 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba18fea-88ef-47f4-95a8-48cdc363aaed None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:58.540944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c313112-b1c5-4b3b-938e-21e9e099c5fa None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:58.541470 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c313112-b1c5-4b3b-938e-21e9e099c5fa None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:58.541940 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c313112-b1c5-4b3b-938e-21e9e099c5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:58.560717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6c313112-b1c5-4b3b-938e-21e9e099c5fa None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:58.562189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6c313112-b1c5-4b3b-938e-21e9e099c5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:58.571900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ec443-367f-4c1a-8a42-faf0f73e5680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:58.572607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ec443-367f-4c1a-8a42-faf0f73e5680 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:58.572865 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ec443-367f-4c1a-8a42-faf0f73e5680 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:58.573146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ec443-367f-4c1a-8a42-faf0f73e5680 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:58.573900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ec443-367f-4c1a-8a42-faf0f73e5680 None admin] RBAC: Adding query filter params (name=member) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:58.574212 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ec443-367f-4c1a-8a42-faf0f73e5680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:58.575633 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ec443-367f-4c1a-8a42-faf0f73e5680 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:58.588958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 128/274] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:48:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:58.590497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ec443-367f-4c1a-8a42-faf0f73e5680 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:58.626640 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:58.652450 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 147/275] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:58.675770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c313112-b1c5-4b3b-938e-21e9e099c5fa None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:58.688532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:58.708490 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ec443-367f-4c1a-8a42-faf0f73e5680 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:58.743716 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02891187-621d-434f-9601-5c354afbacba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:58.744602 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02891187-621d-434f-9601-5c354afbacba None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:58.744889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02891187-621d-434f-9601-5c354afbacba None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:58.745172 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02891187-621d-434f-9601-5c354afbacba None admin] PATH_INFO: `/v3/users/f8ad960128384b369b7fede15342a715` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:58.758944 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02891187-621d-434f-9601-5c354afbacba None admin] RBAC: Authorizing `identity:get_user(user_id=f8ad960128384b369b7fede15342a715)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:58.760444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02891187-621d-434f-9601-5c354afbacba None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:58.762146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 129/276] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:58 2022] GET /identity/v3/users/f8ad960128384b369b7fede15342a715 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:58.763532 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51349743-c5c3-4fd0-9133-773cce069ea5 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:58.763821 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51349743-c5c3-4fd0-9133-773cce069ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:58.764102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51349743-c5c3-4fd0-9133-773cce069ea5 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:58.765476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 130/277] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:58.768644 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c1587-41b0-4a3e-a62f-89db3dee9447 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:58.769153 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c1587-41b0-4a3e-a62f-89db3dee9447 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:58.769728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c1587-41b0-4a3e-a62f-89db3dee9447 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:58.771232 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 131/278] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:48:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:48:58.772077 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c1587-41b0-4a3e-a62f-89db3dee9447 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:58.790291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c313112-b1c5-4b3b-938e-21e9e099c5fa None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:58.825090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5dc310-89b5-4ffe-9356-4ffad8de53a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:58.825877 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dc310-89b5-4ffe-9356-4ffad8de53a8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:58.826160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dc310-89b5-4ffe-9356-4ffad8de53a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:58.826429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dc310-89b5-4ffe-9356-4ffad8de53a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:58.827604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5dc310-89b5-4ffe-9356-4ffad8de53a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:58.829072 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5dc310-89b5-4ffe-9356-4ffad8de53a8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:58.830675 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 148/279] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:48:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:58.831831 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029e64c-ad29-4791-b8d8-3799528bc560 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:58.832114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029e64c-ad29-4791-b8d8-3799528bc560 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:58.832466 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029e64c-ad29-4791-b8d8-3799528bc560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:58.848575 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0029e64c-ad29-4791-b8d8-3799528bc560 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:58.849698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0029e64c-ad29-4791-b8d8-3799528bc560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:58.870090 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:58.877380 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c1587-41b0-4a3e-a62f-89db3dee9447 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:58.911748 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2cb393-29d3-4c5b-8527-f4153e01d29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:58.912404 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cb393-29d3-4c5b-8527-f4153e01d29f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:58.912690 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cb393-29d3-4c5b-8527-f4153e01d29f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:58.912969 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cb393-29d3-4c5b-8527-f4153e01d29f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:58.913790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cb393-29d3-4c5b-8527-f4153e01d29f None admin] RBAC: Adding query filter params (role.id=ffed4493824a4040977d2cd1d057e07b, user.id=f8ad960128384b369b7fede15342a715, scope.system=all) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:58.914105 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cb393-29d3-4c5b-8527-f4153e01d29f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:58.914191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 149/280] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:58.915444 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908152b-3c40-46c9-a975-58f0aaa991cf None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:58.915673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cb393-29d3-4c5b-8527-f4153e01d29f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:58.915766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908152b-3c40-46c9-a975-58f0aaa991cf None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:58.916011 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908152b-3c40-46c9-a975-58f0aaa991cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:58.920729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 132/281] 10.222.0.69 () {56 vars in 1308 bytes} [Thu Jul 14 18:48:58 2022] GET /identity/v3/role_assignments?user.id=f8ad960128384b369b7fede15342a715&scope.system=all&role.id=ffed4493824a4040977d2cd1d057e07b => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:58.921482 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2cb393-29d3-4c5b-8527-f4153e01d29f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:58.941567 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7908152b-3c40-46c9-a975-58f0aaa991cf None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:48:58.943023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7908152b-3c40-46c9-a975-58f0aaa991cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:48:58.945633 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:59.024560 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 150/282] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:48:59.025354 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7908152b-3c40-46c9-a975-58f0aaa991cf None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:59.047591 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2cb393-29d3-4c5b-8527-f4153e01d29f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:59.087947 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865faf3-934e-434d-87ec-d82df287cf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:59.088639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865faf3-934e-434d-87ec-d82df287cf47 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:59.088928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865faf3-934e-434d-87ec-d82df287cf47 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:59.089208 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865faf3-934e-434d-87ec-d82df287cf47 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:59.098431 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865faf3-934e-434d-87ec-d82df287cf47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:59.098431 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865faf3-934e-434d-87ec-d82df287cf47 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:59.143019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7908152b-3c40-46c9-a975-58f0aaa991cf None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:59.178993 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe0422-b896-492a-9032-545808bf0d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:59.179820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe0422-b896-492a-9032-545808bf0d1d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:59.180109 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe0422-b896-492a-9032-545808bf0d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:59.180384 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe0422-b896-492a-9032-545808bf0d1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:59.187306 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:59.191098 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe0422-b896-492a-9032-545808bf0d1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:59.199086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-edfe0422-b896-492a-9032-545808bf0d1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:59.199086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:59.199086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:59.199086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:59.199086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:59.199086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:59.199086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:59.199086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:59.199086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:59.199086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:59.199086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:59.199086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:59.199086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:59.199086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:59.199086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:59.199086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:59.200005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:59.200005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:59.200005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:59.200005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:59.200005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:59.200005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:59.200005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:59.200005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:59.200005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:59.200005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:59.200005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:59.200005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:59.200005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:59.200005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:59.200005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:59.200005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:59.200795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:59.200795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:59.200795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:59.200795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:59.200795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:59.200795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:59.200795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:59.200795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:59.200795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:59.200795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:59.200795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:48:59.200795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:59.200795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 151/283] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:59.201441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe0422-b896-492a-9032-545808bf0d1d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:59.208435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 133/284] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 14 18:48:58 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:48:59.209438 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865faf3-934e-434d-87ec-d82df287cf47 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:59.254747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:59.324373 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe0422-b896-492a-9032-545808bf0d1d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:59.362619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37752c3e-3351-48f4-8b9c-931c66df08ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:59.363514 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37752c3e-3351-48f4-8b9c-931c66df08ce None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:59.363911 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37752c3e-3351-48f4-8b9c-931c66df08ce None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:59.364198 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37752c3e-3351-48f4-8b9c-931c66df08ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:59.369002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865faf3-934e-434d-87ec-d82df287cf47 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:59.376196 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37752c3e-3351-48f4-8b9c-931c66df08ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:48:59.384736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-37752c3e-3351-48f4-8b9c-931c66df08ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 18:48:59.384736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:48:59.384736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:48:59.384736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:48:59.384736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:48:59.384736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:48:59.384736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:48:59.384736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:48:59.384736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:48:59.384736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:48:59.384736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:48:59.384736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:48:59.384736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:48:59.384736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:48:59.384736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:48:59.384736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:48:59.385691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:48:59.385691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:48:59.385691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:48:59.385691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:48:59.385691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:48:59.385691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:48:59.385691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:48:59.385691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:48:59.385691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:48:59.385691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:48:59.385691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:48:59.385691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:48:59.385691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:48:59.385691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:48:59.385691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:48:59.385691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:48:59.386456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:48:59.386456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:48:59.386456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:48:59.386456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:48:59.386456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:48:59.386456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:48:59.386456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:48:59.386456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:48:59.386456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:48:59.386456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:48:59.386456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 18:48:59.386456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:48:59.386456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 152/285] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:48:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:48:59.392771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37752c3e-3351-48f4-8b9c-931c66df08ce None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:59.415609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4461918-b376-407d-b54e-7f7006203c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:59.416648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4461918-b376-407d-b54e-7f7006203c8c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:59.417317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4461918-b376-407d-b54e-7f7006203c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:59.417666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4461918-b376-407d-b54e-7f7006203c8c None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:59.418903 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4461918-b376-407d-b54e-7f7006203c8c None admin] RBAC: Adding query filter params (name=member) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:59.419305 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4461918-b376-407d-b54e-7f7006203c8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:59.420761 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4461918-b376-407d-b54e-7f7006203c8c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:59.440748 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 134/286] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:48:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:59.451142 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4461918-b376-407d-b54e-7f7006203c8c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:59.462537 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:59.516613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37752c3e-3351-48f4-8b9c-931c66df08ce None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:59.557344 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:59.557921 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e4393-1dfe-4864-a20c-68f74561bf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:59.558613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e4393-1dfe-4864-a20c-68f74561bf87 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:59.558964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e4393-1dfe-4864-a20c-68f74561bf87 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:59.559202 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e4393-1dfe-4864-a20c-68f74561bf87 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:59.560032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e4393-1dfe-4864-a20c-68f74561bf87 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:48:59.560396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e4393-1dfe-4864-a20c-68f74561bf87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:59.561810 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e4393-1dfe-4864-a20c-68f74561bf87 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:59.572099 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 153/287] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:48:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:59.577695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e4393-1dfe-4864-a20c-68f74561bf87 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:59.588530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4461918-b376-407d-b54e-7f7006203c8c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:59.631230 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150467bd-3def-4994-8357-9a0948d9476a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:59.632637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150467bd-3def-4994-8357-9a0948d9476a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:59.633118 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150467bd-3def-4994-8357-9a0948d9476a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:59.633554 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150467bd-3def-4994-8357-9a0948d9476a None admin] PATH_INFO: `/v3/users/3abf085883f14d268b61efb2618311a5` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:59.634981 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150467bd-3def-4994-8357-9a0948d9476a None admin] RBAC: Authorizing `identity:get_user(user_id=3abf085883f14d268b61efb2618311a5)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:59.636736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150467bd-3def-4994-8357-9a0948d9476a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:59.638776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 135/288] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:59 2022] GET /identity/v3/users/3abf085883f14d268b61efb2618311a5 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:48:59.644823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150467bd-3def-4994-8357-9a0948d9476a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:59.718505 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e4393-1dfe-4864-a20c-68f74561bf87 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:59.745608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:59.770816 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f49a5d-28ca-4641-a61e-4828c2e532f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:59.772408 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f49a5d-28ca-4641-a61e-4828c2e532f1 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:59.772854 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f49a5d-28ca-4641-a61e-4828c2e532f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:59.773269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f49a5d-28ca-4641-a61e-4828c2e532f1 None admin] PATH_INFO: `/v3/users/22d68157f8464127a6a3a26814cb4f36` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:59.775061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f49a5d-28ca-4641-a61e-4828c2e532f1 None admin] RBAC: Authorizing `identity:get_user(user_id=22d68157f8464127a6a3a26814cb4f36)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:59.777938 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f49a5d-28ca-4641-a61e-4828c2e532f1 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:59.781117 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 154/289] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:48:59 2022] GET /identity/v3/users/22d68157f8464127a6a3a26814cb4f36 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:59.786342 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f49a5d-28ca-4641-a61e-4828c2e532f1 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:48:59.794881 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150467bd-3def-4994-8357-9a0948d9476a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:59.881940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94506727-bf4e-4b4c-aca9-e9a040fe3810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:59.882634 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94506727-bf4e-4b4c-aca9-e9a040fe3810 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:59.882989 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94506727-bf4e-4b4c-aca9-e9a040fe3810 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:59.883362 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94506727-bf4e-4b4c-aca9-e9a040fe3810 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:59.910561 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f49a5d-28ca-4641-a61e-4828c2e532f1 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:48:59.916589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:48:59.951010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73339a12-efd1-4e48-b70e-c74e6dcf6cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:48:59.952699 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73339a12-efd1-4e48-b70e-c74e6dcf6cee None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:48:59.953195 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73339a12-efd1-4e48-b70e-c74e6dcf6cee None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:48:59.953652 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73339a12-efd1-4e48-b70e-c74e6dcf6cee None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:48:59.955265 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73339a12-efd1-4e48-b70e-c74e6dcf6cee None admin] RBAC: Authorizing `identity:get_project(project_id=45bac146c9d842bf82503d3f2b145128)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:48:59.958183 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73339a12-efd1-4e48-b70e-c74e6dcf6cee None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:48:59.960461 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 155/290] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:48:59 2022] GET /identity/v3/projects/45bac146c9d842bf82503d3f2b145128 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:48:59.968135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73339a12-efd1-4e48-b70e-c74e6dcf6cee None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:00.041933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94506727-bf4e-4b4c-aca9-e9a040fe3810 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:00.043605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94506727-bf4e-4b4c-aca9-e9a040fe3810 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:00.127530 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:00.161067 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 136/291] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:48:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:00.166356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94506727-bf4e-4b4c-aca9-e9a040fe3810 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:00.217184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73339a12-efd1-4e48-b70e-c74e6dcf6cee None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:00.231189 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:00.271148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684d0a5-475e-4c4b-b093-29e9890e06a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:00.272449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684d0a5-475e-4c4b-b093-29e9890e06a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:00.272961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684d0a5-475e-4c4b-b093-29e9890e06a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:00.273450 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684d0a5-475e-4c4b-b093-29e9890e06a2 None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128/users/22d68157f8464127a6a3a26814cb4f36/roles/a2403691e5c440df989bc6aad7118272` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:00.282996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684d0a5-475e-4c4b-b093-29e9890e06a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=45bac146c9d842bf82503d3f2b145128, user_id=22d68157f8464127a6a3a26814cb4f36, role_id=a2403691e5c440df989bc6aad7118272)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:00.282996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684d0a5-475e-4c4b-b093-29e9890e06a2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:00.283246 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94506727-bf4e-4b4c-aca9-e9a040fe3810 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:00.317886 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eeec5e-4e5d-42cf-867f-c1ef3ee9332b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:00.318963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eeec5e-4e5d-42cf-867f-c1ef3ee9332b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:00.319287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eeec5e-4e5d-42cf-867f-c1ef3ee9332b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:00.319573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eeec5e-4e5d-42cf-867f-c1ef3ee9332b None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:00.320855 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eeec5e-4e5d-42cf-867f-c1ef3ee9332b None admin] RBAC: Authorizing `identity:get_project(project_id=45bac146c9d842bf82503d3f2b145128)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:00.322432 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eeec5e-4e5d-42cf-867f-c1ef3ee9332b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:00.324580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 137/292] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:49:00 2022] GET /identity/v3/projects/45bac146c9d842bf82503d3f2b145128 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:00.330344 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eeec5e-4e5d-42cf-867f-c1ef3ee9332b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:00.389674 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 156/293] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:48:59 2022] PUT /identity/v3/projects/45bac146c9d842bf82503d3f2b145128/users/22d68157f8464127a6a3a26814cb4f36/roles/a2403691e5c440df989bc6aad7118272 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:49:00.468695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eeec5e-4e5d-42cf-867f-c1ef3ee9332b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:00.490409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:00.527109 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19433329-7404-4973-86f4-1f4b999ee04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:00.527815 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19433329-7404-4973-86f4-1f4b999ee04a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:00.528100 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19433329-7404-4973-86f4-1f4b999ee04a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:00.528378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19433329-7404-4973-86f4-1f4b999ee04a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:00.529232 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19433329-7404-4973-86f4-1f4b999ee04a None admin] RBAC: Adding query filter params (role.id=ffed4493824a4040977d2cd1d057e07b, scope.project.id=45bac146c9d842bf82503d3f2b145128, user.id=3abf085883f14d268b61efb2618311a5) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:00.529544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19433329-7404-4973-86f4-1f4b999ee04a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:00.531019 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19433329-7404-4973-86f4-1f4b999ee04a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:00.540255 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 138/294] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:49:00 2022] GET /identity/v3/role_assignments?user.id=3abf085883f14d268b61efb2618311a5&scope.project.id=45bac146c9d842bf82503d3f2b145128&role.id=ffed4493824a4040977d2cd1d057e07b => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:01.040950 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdee9d-dfc9-44e1-b0bd-360aa2863cdc None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:01.041262 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdee9d-dfc9-44e1-b0bd-360aa2863cdc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:01.041529 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdee9d-dfc9-44e1-b0bd-360aa2863cdc None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:01.043190 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 157/295] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:01.048522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc209d-da37-4c9f-9c42-10e4f9063c11 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:01.048822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc209d-da37-4c9f-9c42-10e4f9063c11 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:01.049111 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc209d-da37-4c9f-9c42-10e4f9063c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:01.068656 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-5ddc209d-da37-4c9f-9c42-10e4f9063c11 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:01.069853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5ddc209d-da37-4c9f-9c42-10e4f9063c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:01.140242 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 139/296] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:01.162982 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdee9d-dfc9-44e1-b0bd-360aa2863cdc None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:01.270396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdee9d-dfc9-44e1-b0bd-360aa2863cdc None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:01.287823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:01.306286 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30e0f8-bb86-4362-9e9e-cf469a3c375b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:01.307255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30e0f8-bb86-4362-9e9e-cf469a3c375b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:01.307465 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30e0f8-bb86-4362-9e9e-cf469a3c375b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:01.307848 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30e0f8-bb86-4362-9e9e-cf469a3c375b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:01.319874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30e0f8-bb86-4362-9e9e-cf469a3c375b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:01.327956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-6e30e0f8-bb86-4362-9e9e-cf469a3c375b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:49:01.327956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:01.327956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:01.327956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:01.327956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:01.327956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:01.327956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:01.327956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:01.327956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:01.327956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:01.327956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:01.327956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:01.327956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:01.327956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:01.327956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:01.327956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:01.328853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:01.328853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:01.328853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:01.328853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:01.328853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:01.328853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:01.328853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:01.328853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:01.328853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:01.328853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:01.328853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:01.328853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:01.328853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:01.328853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:01.328853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:01.328853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:01.329619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:01.329619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:01.329619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:01.329619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:01.329619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:01.329619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:01.329619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:01.329619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:01.329619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:01.329619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:01.329619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:49:01.329619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:01.330201 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 158/297] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:01.336185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc209d-da37-4c9f-9c42-10e4f9063c11 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:01.443378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc209d-da37-4c9f-9c42-10e4f9063c11 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:01.451706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:01.478916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe84b8c-d2bf-4377-8cc7-5453e15917c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:01.479932 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe84b8c-d2bf-4377-8cc7-5453e15917c4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:01.480445 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe84b8c-d2bf-4377-8cc7-5453e15917c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:01.480923 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe84b8c-d2bf-4377-8cc7-5453e15917c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:01.481865 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe84b8c-d2bf-4377-8cc7-5453e15917c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:01.482424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe84b8c-d2bf-4377-8cc7-5453e15917c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:01.484053 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe84b8c-d2bf-4377-8cc7-5453e15917c4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:01.493599 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 140/298] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:49:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:01.500593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30e0f8-bb86-4362-9e9e-cf469a3c375b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:01.605612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30e0f8-bb86-4362-9e9e-cf469a3c375b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:01.641812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de2e76-fd83-42e6-98a8-b42bb576fdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:01.642694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de2e76-fd83-42e6-98a8-b42bb576fdbf None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:01.642981 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de2e76-fd83-42e6-98a8-b42bb576fdbf None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:01.643288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de2e76-fd83-42e6-98a8-b42bb576fdbf None admin] PATH_INFO: `/v3/users/f8ad960128384b369b7fede15342a715` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:01.644571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de2e76-fd83-42e6-98a8-b42bb576fdbf None admin] RBAC: Authorizing `identity:get_user(user_id=f8ad960128384b369b7fede15342a715)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:01.646225 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de2e76-fd83-42e6-98a8-b42bb576fdbf None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:01.648229 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 159/299] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:49:01 2022] GET /identity/v3/users/f8ad960128384b369b7fede15342a715 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:01.654127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe84b8c-d2bf-4377-8cc7-5453e15917c4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:01.681761 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42256368-8240-4dcd-82f3-530176cdac0c None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:01.682055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42256368-8240-4dcd-82f3-530176cdac0c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:01.682393 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42256368-8240-4dcd-82f3-530176cdac0c None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:01.684071 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 160/300] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:01.688437 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04079b-92dc-452e-98bc-f11bd3a85f3e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:01.688752 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04079b-92dc-452e-98bc-f11bd3a85f3e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:01.689016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04079b-92dc-452e-98bc-f11bd3a85f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:01.711189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-cd04079b-92dc-452e-98bc-f11bd3a85f3e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:01.711189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cd04079b-92dc-452e-98bc-f11bd3a85f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:01.753642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:01.786131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe84b8c-d2bf-4377-8cc7-5453e15917c4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:01.786454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 161/301] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:49:01.810516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04079b-92dc-452e-98bc-f11bd3a85f3e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:01.834847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f3d83a-0a2a-4c5e-8448-5be674d5834f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:01.835749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3d83a-0a2a-4c5e-8448-5be674d5834f None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:01.836038 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3d83a-0a2a-4c5e-8448-5be674d5834f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:01.836324 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3d83a-0a2a-4c5e-8448-5be674d5834f None admin] PATH_INFO: `/v3/system/users/f8ad960128384b369b7fede15342a715/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:01.837935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f3d83a-0a2a-4c5e-8448-5be674d5834f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f8ad960128384b369b7fede15342a715, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:01.839686 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f3d83a-0a2a-4c5e-8448-5be674d5834f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:01.917310 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04079b-92dc-452e-98bc-f11bd3a85f3e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:01.936233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 141/302] 10.222.0.69 () {58 vars in 1334 bytes} [Thu Jul 14 18:49:01 2022] PUT /identity/v3/system/users/f8ad960128384b369b7fede15342a715/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:49:01.937945 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612ed97-c85c-4780-a65c-4d2217271d61 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:01.938365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612ed97-c85c-4780-a65c-4d2217271d61 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:01.938710 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612ed97-c85c-4780-a65c-4d2217271d61 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:01.940181 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 142/303] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:01.947005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f9503-ea6e-429d-86f0-f066883b12d2 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:01.947273 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f9503-ea6e-429d-86f0-f066883b12d2 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:01.947549 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f9503-ea6e-429d-86f0-f066883b12d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:01.958930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a98d48-79e3-4154-a6c2-7fb279bb3540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:01.959797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a98d48-79e3-4154-a6c2-7fb279bb3540 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:01.960188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a98d48-79e3-4154-a6c2-7fb279bb3540 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:01.960484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a98d48-79e3-4154-a6c2-7fb279bb3540 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:01.966213 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-fe7f9503-ea6e-429d-86f0-f066883b12d2 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:01.967600 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-fe7f9503-ea6e-429d-86f0-f066883b12d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:01.973113 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a98d48-79e3-4154-a6c2-7fb279bb3540 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:01.981906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-a6a98d48-79e3-4154-a6c2-7fb279bb3540 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 18:49:01.981906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:01.981906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:01.981906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:01.981906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:01.981906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:01.981906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:01.981906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:01.981906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:01.981906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:01.981906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:01.981906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:01.981906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:01.981906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:01.981906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:01.981906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:01.982858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:01.982858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:01.982858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:01.982858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:01.982858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:01.982858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:01.982858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:01.982858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:01.982858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:01.982858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:01.982858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:01.982858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:01.982858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:01.982858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:01.982858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:01.982858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:01.983669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:01.983669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:01.983669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:01.983669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:01.983669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:01.983669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:01.983669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:01.983669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:01.983669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:01.983669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:01.983669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 18:49:01.983669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:01.983669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 162/304] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:49:01.996301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a98d48-79e3-4154-a6c2-7fb279bb3540 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:02.004704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:02.043774 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:02.075636 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 143/305] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:49:02.094835 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f9503-ea6e-429d-86f0-f066883b12d2 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:02.128148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a98d48-79e3-4154-a6c2-7fb279bb3540 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:02.182506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f14a9a-09cc-4a93-8df6-3990d5d634c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:02.183600 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f14a9a-09cc-4a93-8df6-3990d5d634c6 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:02.183904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f14a9a-09cc-4a93-8df6-3990d5d634c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:02.184189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f14a9a-09cc-4a93-8df6-3990d5d634c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:02.184954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f14a9a-09cc-4a93-8df6-3990d5d634c6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:02.185270 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f14a9a-09cc-4a93-8df6-3990d5d634c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:02.187542 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f14a9a-09cc-4a93-8df6-3990d5d634c6 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:02.204278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 163/306] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:49:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:02.205357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f14a9a-09cc-4a93-8df6-3990d5d634c6 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:02.214977 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f9503-ea6e-429d-86f0-f066883b12d2 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:02.260726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6238d-be59-4c36-bd84-086d8ccc1a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:02.261540 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6238d-be59-4c36-bd84-086d8ccc1a41 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:02.261825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6238d-be59-4c36-bd84-086d8ccc1a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:02.262103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6238d-be59-4c36-bd84-086d8ccc1a41 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:02.264030 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6238d-be59-4c36-bd84-086d8ccc1a41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:02.266406 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6238d-be59-4c36-bd84-086d8ccc1a41 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:02.268384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 144/307] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:02.273915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6238d-be59-4c36-bd84-086d8ccc1a41 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:02.306885 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:02.337879 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:02.345658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f14a9a-09cc-4a93-8df6-3990d5d634c6 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:02.389936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d045c30-fcc4-4aa0-8953-79cfd13898d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:02.390182 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6238d-be59-4c36-bd84-086d8ccc1a41 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:02.391065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d045c30-fcc4-4aa0-8953-79cfd13898d4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:02.391571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d045c30-fcc4-4aa0-8953-79cfd13898d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:02.391952 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d045c30-fcc4-4aa0-8953-79cfd13898d4 None admin] PATH_INFO: `/v3/users/ee92c2e937124bcca9b7fe6f61aed1b9` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:02.406894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d045c30-fcc4-4aa0-8953-79cfd13898d4 None admin] RBAC: Authorizing `identity:get_user(user_id=ee92c2e937124bcca9b7fe6f61aed1b9)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:02.408657 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d045c30-fcc4-4aa0-8953-79cfd13898d4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:02.411137 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 164/308] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:49:02 2022] GET /identity/v3/users/ee92c2e937124bcca9b7fe6f61aed1b9 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:02.416898 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d045c30-fcc4-4aa0-8953-79cfd13898d4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:02.431895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbacf7-5324-48a7-b4d1-a94336f917f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:02.432818 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbacf7-5324-48a7-b4d1-a94336f917f9 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:02.433319 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbacf7-5324-48a7-b4d1-a94336f917f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:02.433598 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbacf7-5324-48a7-b4d1-a94336f917f9 None admin] PATH_INFO: `/v3/groups` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:02.434429 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbacf7-5324-48a7-b4d1-a94336f917f9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:02.436086 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbacf7-5324-48a7-b4d1-a94336f917f9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:02.511310 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 145/309] 10.222.0.69 () {60 vars in 1134 bytes} [Thu Jul 14 18:49:02 2022] POST /identity/v3/groups => generated 229 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:49:02.535802 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d045c30-fcc4-4aa0-8953-79cfd13898d4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:02.578454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84670c-1149-4ca6-a1ab-fa3f09e17fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:02.579470 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84670c-1149-4ca6-a1ab-fa3f09e17fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:02.579897 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84670c-1149-4ca6-a1ab-fa3f09e17fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:02.580313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84670c-1149-4ca6-a1ab-fa3f09e17fe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:02.581379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84670c-1149-4ca6-a1ab-fa3f09e17fe9 None admin] RBAC: Adding query filter params (role.id=a2403691e5c440df989bc6aad7118272, user.id=ee92c2e937124bcca9b7fe6f61aed1b9, scope.system=all) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:02.581894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84670c-1149-4ca6-a1ab-fa3f09e17fe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:02.583614 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84670c-1149-4ca6-a1ab-fa3f09e17fe9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:02.590599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 165/310] 10.222.0.69 () {56 vars in 1308 bytes} [Thu Jul 14 18:49:02 2022] GET /identity/v3/role_assignments?user.id=ee92c2e937124bcca9b7fe6f61aed1b9&scope.system=all&role.id=a2403691e5c440df989bc6aad7118272 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:02.730805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69f66a-3b13-48f6-ae59-096ae6e2ea5b None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:02.731002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69f66a-3b13-48f6-ae59-096ae6e2ea5b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:02.731338 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69f66a-3b13-48f6-ae59-096ae6e2ea5b None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:02.733039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 146/311] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:02.739401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc8029-253a-4769-8862-1ea3078cbe3d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:02.739701 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc8029-253a-4769-8862-1ea3078cbe3d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:02.739988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc8029-253a-4769-8862-1ea3078cbe3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:02.758708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-06cc8029-253a-4769-8862-1ea3078cbe3d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:02.759964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-06cc8029-253a-4769-8862-1ea3078cbe3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:02.777242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:02.833299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 166/312] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:02.856954 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69f66a-3b13-48f6-ae59-096ae6e2ea5b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:02.960740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:02.971465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69f66a-3b13-48f6-ae59-096ae6e2ea5b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:03.006808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23180e0-882a-4b5e-a636-c4a607cabdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:03.007570 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23180e0-882a-4b5e-a636-c4a607cabdc2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:03.007858 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23180e0-882a-4b5e-a636-c4a607cabdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:03.008140 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23180e0-882a-4b5e-a636-c4a607cabdc2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:03.019070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23180e0-882a-4b5e-a636-c4a607cabdc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:03.028228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-a23180e0-882a-4b5e-a636-c4a607cabdc2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 18:49:03.028228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:03.028228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:03.028228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:03.028228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:03.028228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:03.028228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:03.028228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:03.028228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:03.028228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:03.028228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:03.028228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:03.028228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:03.028228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:03.028228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:03.028228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:03.029215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:03.029215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:03.029215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:03.029215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:03.029215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:03.029215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:03.029215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:03.029215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:03.029215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:03.029215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:03.029215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:03.029215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:03.029215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:03.029215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:03.029215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:03.029215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:03.029983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:03.029983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:03.029983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:03.029983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:03.029983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:03.029983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:03.029983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:03.029983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:03.029983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:03.029983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:03.029983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 18:49:03.029983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:03.032676 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 147/313] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:03.039187 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc8029-253a-4769-8862-1ea3078cbe3d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:03.150270 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc8029-253a-4769-8862-1ea3078cbe3d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:03.187429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14756e67-e37a-4ed3-bfc6-643f64765390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:03.188162 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14756e67-e37a-4ed3-bfc6-643f64765390 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:03.188456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14756e67-e37a-4ed3-bfc6-643f64765390 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:03.188737 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14756e67-e37a-4ed3-bfc6-643f64765390 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:03.189486 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14756e67-e37a-4ed3-bfc6-643f64765390 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:03.189796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14756e67-e37a-4ed3-bfc6-643f64765390 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:03.191209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14756e67-e37a-4ed3-bfc6-643f64765390 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:03.200334 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 167/314] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:49:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:03.208868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23180e0-882a-4b5e-a636-c4a607cabdc2 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:03.312711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23180e0-882a-4b5e-a636-c4a607cabdc2 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:03.351222 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7c9f4-6cf8-431b-b2d6-0c72c9ee6b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:03.352114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7c9f4-6cf8-431b-b2d6-0c72c9ee6b78 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:03.352402 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7c9f4-6cf8-431b-b2d6-0c72c9ee6b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:03.352681 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7c9f4-6cf8-431b-b2d6-0c72c9ee6b78 None admin] PATH_INFO: `/v3/users/22d68157f8464127a6a3a26814cb4f36` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:03.353946 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7c9f4-6cf8-431b-b2d6-0c72c9ee6b78 None admin] RBAC: Authorizing `identity:get_user(user_id=22d68157f8464127a6a3a26814cb4f36)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:03.355519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7c9f4-6cf8-431b-b2d6-0c72c9ee6b78 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:03.357466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 148/315] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:49:03 2022] GET /identity/v3/users/22d68157f8464127a6a3a26814cb4f36 => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:03.364462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14756e67-e37a-4ed3-bfc6-643f64765390 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:03.377228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:03.467787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14756e67-e37a-4ed3-bfc6-643f64765390 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:03.501194 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a57f2-1e81-469a-b4de-0f302f718d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:03.501812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a57f2-1e81-469a-b4de-0f302f718d08 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:03.502103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a57f2-1e81-469a-b4de-0f302f718d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:03.502386 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a57f2-1e81-469a-b4de-0f302f718d08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:03.576213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a57f2-1e81-469a-b4de-0f302f718d08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:03.577693 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a57f2-1e81-469a-b4de-0f302f718d08 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:03.591955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:03.680660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 168/316] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:03.686307 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7c9f4-6cf8-431b-b2d6-0c72c9ee6b78 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:03.706902 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:03.792075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7c9f4-6cf8-431b-b2d6-0c72c9ee6b78 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:03.836335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c9c3-164d-4b6a-a7fa-61cd96cea6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:03.837545 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c9c3-164d-4b6a-a7fa-61cd96cea6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:03.838047 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c9c3-164d-4b6a-a7fa-61cd96cea6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:03.838515 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c9c3-164d-4b6a-a7fa-61cd96cea6e6 None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:03.839902 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15c9c3-164d-4b6a-a7fa-61cd96cea6e6 None admin] RBAC: Authorizing `identity:get_project(project_id=45bac146c9d842bf82503d3f2b145128)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:03.841553 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15c9c3-164d-4b6a-a7fa-61cd96cea6e6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:03.843492 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 149/317] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:49:03 2022] GET /identity/v3/projects/45bac146c9d842bf82503d3f2b145128 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:03.849867 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a57f2-1e81-469a-b4de-0f302f718d08 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:03.951446 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a57f2-1e81-469a-b4de-0f302f718d08 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:03.962442 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:03.986058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91b95f-572c-4478-a954-7d983d011a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:03.986724 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91b95f-572c-4478-a954-7d983d011a0d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:03.987015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91b95f-572c-4478-a954-7d983d011a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:03.987331 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91b95f-572c-4478-a954-7d983d011a0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:03.988165 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91b95f-572c-4478-a954-7d983d011a0d None admin] RBAC: Adding query filter params (role.id=a2403691e5c440df989bc6aad7118272, scope.project.id=45bac146c9d842bf82503d3f2b145128, user.id=22d68157f8464127a6a3a26814cb4f36) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:03.988585 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91b95f-572c-4478-a954-7d983d011a0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:03.990036 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91b95f-572c-4478-a954-7d983d011a0d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:03.996521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 169/318] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:49:03 2022] GET /identity/v3/role_assignments?user.id=22d68157f8464127a6a3a26814cb4f36&scope.project.id=45bac146c9d842bf82503d3f2b145128&role.id=a2403691e5c440df989bc6aad7118272 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:04.152935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7ea9d-1439-41ac-8a35-01379eb8af7b None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:04.153274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7ea9d-1439-41ac-8a35-01379eb8af7b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:04.153465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7ea9d-1439-41ac-8a35-01379eb8af7b None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:04.154953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 150/319] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:04.160319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332b3d0-696f-4eea-b1c3-79d2fe18049b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:04.160692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332b3d0-696f-4eea-b1c3-79d2fe18049b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:04.160990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332b3d0-696f-4eea-b1c3-79d2fe18049b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:04.178749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5332b3d0-696f-4eea-b1c3-79d2fe18049b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:04.179992 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5332b3d0-696f-4eea-b1c3-79d2fe18049b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:04.251685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 170/320] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:04.276319 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7ea9d-1439-41ac-8a35-01379eb8af7b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:04.361977 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:04.377790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7ea9d-1439-41ac-8a35-01379eb8af7b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:04.413528 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c877d-d512-48e8-93b6-b9ff504b02d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:04.414251 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c877d-d512-48e8-93b6-b9ff504b02d7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:04.414537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c877d-d512-48e8-93b6-b9ff504b02d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:04.414814 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c877d-d512-48e8-93b6-b9ff504b02d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:04.425200 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82c877d-d512-48e8-93b6-b9ff504b02d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:04.433563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b82c877d-d512-48e8-93b6-b9ff504b02d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:49:04.433563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:04.433563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:04.433563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:04.433563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:04.433563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:04.433563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:04.433563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:04.433563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:04.433563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:04.433563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:04.433563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:04.433563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:04.433563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:04.433563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:04.433563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:04.434441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:04.434441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:04.434441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:04.434441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:04.434441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:04.434441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:04.434441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:04.434441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:04.434441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:04.434441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:04.434441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:04.434441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:04.434441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:04.434441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:04.434441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:04.434441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:04.435229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:04.435229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:04.435229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:04.435229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:04.435229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:04.435229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:04.435229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:04.435229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:04.435229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:04.435229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:04.435229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:49:04.435229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:04.435229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 151/321] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:04.440566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332b3d0-696f-4eea-b1c3-79d2fe18049b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:04.558579 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332b3d0-696f-4eea-b1c3-79d2fe18049b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:04.560806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfa59e-d7c9-40bc-a461-48e9eac8f20f None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:04.561307 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:04.561424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfa59e-d7c9-40bc-a461-48e9eac8f20f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:04.561424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfa59e-d7c9-40bc-a461-48e9eac8f20f None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:04.562872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 152/322] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:49:04.568546 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d296b4-77b3-4242-a3af-f7133c660a5f None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:04.572991 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d296b4-77b3-4242-a3af-f7133c660a5f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:04.572991 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d296b4-77b3-4242-a3af-f7133c660a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:04.585907 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-93d296b4-77b3-4242-a3af-f7133c660a5f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:04.587104 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-93d296b4-77b3-4242-a3af-f7133c660a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:04.595953 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cde1b6-4f2a-4962-b36e-c3532568e5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:04.596730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cde1b6-4f2a-4962-b36e-c3532568e5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:04.597016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cde1b6-4f2a-4962-b36e-c3532568e5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:04.597301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cde1b6-4f2a-4962-b36e-c3532568e5a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:04.598125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cde1b6-4f2a-4962-b36e-c3532568e5a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:04.598332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cde1b6-4f2a-4962-b36e-c3532568e5a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:04.599860 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cde1b6-4f2a-4962-b36e-c3532568e5a1 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:04.609022 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 171/323] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:49:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:04.614840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cde1b6-4f2a-4962-b36e-c3532568e5a1 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:04.656012 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 153/324] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:49:04.678081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d296b4-77b3-4242-a3af-f7133c660a5f None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:04.717597 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cde1b6-4f2a-4962-b36e-c3532568e5a1 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:04.751767 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0227182-d693-45c3-a7dc-96a7c46727a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:04.752650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0227182-d693-45c3-a7dc-96a7c46727a2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:04.752934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0227182-d693-45c3-a7dc-96a7c46727a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:04.753208 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0227182-d693-45c3-a7dc-96a7c46727a2 None admin] PATH_INFO: `/v3/users/f8ad960128384b369b7fede15342a715` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:04.754946 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0227182-d693-45c3-a7dc-96a7c46727a2 None admin] RBAC: Authorizing `identity:get_user(user_id=f8ad960128384b369b7fede15342a715)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:04.756426 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0227182-d693-45c3-a7dc-96a7c46727a2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:04.756725 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:04.758078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 172/325] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:49:04 2022] GET /identity/v3/users/f8ad960128384b369b7fede15342a715 => generated 293 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:04.760342 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64ca4a-16c2-495f-ba7e-0d1b51a675f3 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:04.760651 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64ca4a-16c2-495f-ba7e-0d1b51a675f3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:04.760913 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64ca4a-16c2-495f-ba7e-0d1b51a675f3 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:04.762259 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 173/326] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:04.764034 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64ca4a-16c2-495f-ba7e-0d1b51a675f3 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:04.779629 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d296b4-77b3-4242-a3af-f7133c660a5f None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:04.799263 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:04.814184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ad3a9-3b58-4524-9381-75d15afd489c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:04.815059 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ad3a9-3b58-4524-9381-75d15afd489c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:04.815433 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ad3a9-3b58-4524-9381-75d15afd489c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:04.815774 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ad3a9-3b58-4524-9381-75d15afd489c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:04.817023 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ad3a9-3b58-4524-9381-75d15afd489c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:04.818540 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ad3a9-3b58-4524-9381-75d15afd489c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:04.820197 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 154/327] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:04.821319 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bac9e0-eb73-45ae-86cd-aa56c6c67f2d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:04.821604 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bac9e0-eb73-45ae-86cd-aa56c6c67f2d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:04.821880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bac9e0-eb73-45ae-86cd-aa56c6c67f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:04.839032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-78bac9e0-eb73-45ae-86cd-aa56c6c67f2d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:04.840226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-78bac9e0-eb73-45ae-86cd-aa56c6c67f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:04.865427 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64ca4a-16c2-495f-ba7e-0d1b51a675f3 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:04.897873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff89524-3420-47f3-8b44-85a373ff4a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:04.898525 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff89524-3420-47f3-8b44-85a373ff4a3c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:04.898810 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff89524-3420-47f3-8b44-85a373ff4a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:04.899084 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff89524-3420-47f3-8b44-85a373ff4a3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:04.899931 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff89524-3420-47f3-8b44-85a373ff4a3c None admin] RBAC: Adding query filter params (role.id=ffed4493824a4040977d2cd1d057e07b, user.id=f8ad960128384b369b7fede15342a715, scope.system=all) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:04.900243 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff89524-3420-47f3-8b44-85a373ff4a3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:04.901956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff89524-3420-47f3-8b44-85a373ff4a3c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:04.905578 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 155/328] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:04.906377 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bac9e0-eb73-45ae-86cd-aa56c6c67f2d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:04.915378 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 174/329] 10.222.0.69 () {56 vars in 1308 bytes} [Thu Jul 14 18:49:04 2022] GET /identity/v3/role_assignments?user.id=f8ad960128384b369b7fede15342a715&scope.system=all&role.id=ffed4493824a4040977d2cd1d057e07b => generated 510 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:04.932828 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff89524-3420-47f3-8b44-85a373ff4a3c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:04.993800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:05.020660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bac9e0-eb73-45ae-86cd-aa56c6c67f2d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:05.046634 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:05.058874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff89524-3420-47f3-8b44-85a373ff4a3c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:05.060950 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af08fa-771c-4465-93a7-03fb724de4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:05.062335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af08fa-771c-4465-93a7-03fb724de4d2 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:05.062694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af08fa-771c-4465-93a7-03fb724de4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:05.063039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af08fa-771c-4465-93a7-03fb724de4d2 None admin] PATH_INFO: `/v3/groups` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:05.065197 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af08fa-771c-4465-93a7-03fb724de4d2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:05.066888 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af08fa-771c-4465-93a7-03fb724de4d2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:05.095726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b90cc-4684-4030-b488-59d28cfdb72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:05.096620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b90cc-4684-4030-b488-59d28cfdb72d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:05.096912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b90cc-4684-4030-b488-59d28cfdb72d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:05.097193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b90cc-4684-4030-b488-59d28cfdb72d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:05.108326 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b90cc-4684-4030-b488-59d28cfdb72d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:05.117096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-8e2b90cc-4684-4030-b488-59d28cfdb72d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 18:49:05.117096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:05.117096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:05.117096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:05.117096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:05.117096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:05.117096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:05.117096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:05.117096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:05.117096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:05.117096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:05.117096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:05.117096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:05.117096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:05.117096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:05.117096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:05.121630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:05.121630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:05.121630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:05.121630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:05.121630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:05.121630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:05.121630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:05.121630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:05.121630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:05.121630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:05.121630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:05.121630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:05.121630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:05.121630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:05.121630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:05.121630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:05.122593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:05.122593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:05.122593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:05.122593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:05.122593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:05.122593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:05.122593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:05.122593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:05.122593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:05.122593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:05.122593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 18:49:05.122593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:05.122593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 175/330] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:49:05.124104 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b90cc-4684-4030-b488-59d28cfdb72d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:05.145858 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 156/331] 10.222.0.69 () {60 vars in 1134 bytes} [Thu Jul 14 18:49:04 2022] POST /identity/v3/groups => generated 226 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:49:05.251233 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b90cc-4684-4030-b488-59d28cfdb72d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:05.287589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3f9bc-380d-4ea9-a213-a800fba6d852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:05.288272 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3f9bc-380d-4ea9-a213-a800fba6d852 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:05.288566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3f9bc-380d-4ea9-a213-a800fba6d852 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:05.288843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3f9bc-380d-4ea9-a213-a800fba6d852 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:05.289593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3f9bc-380d-4ea9-a213-a800fba6d852 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:05.289903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3f9bc-380d-4ea9-a213-a800fba6d852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:05.291313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3f9bc-380d-4ea9-a213-a800fba6d852 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:05.300377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 176/332] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:49:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:05.306229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af08fa-771c-4465-93a7-03fb724de4d2 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:05.412454 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af08fa-771c-4465-93a7-03fb724de4d2 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:05.450151 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46911ca0-01b4-4319-b350-872e2bbb9836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:05.451051 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46911ca0-01b4-4319-b350-872e2bbb9836 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:05.451426 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46911ca0-01b4-4319-b350-872e2bbb9836 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:05.451706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46911ca0-01b4-4319-b350-872e2bbb9836 None admin] PATH_INFO: `/v3/users/ee92c2e937124bcca9b7fe6f61aed1b9` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:05.452959 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46911ca0-01b4-4319-b350-872e2bbb9836 None admin] RBAC: Authorizing `identity:get_user(user_id=ee92c2e937124bcca9b7fe6f61aed1b9)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:05.454446 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46911ca0-01b4-4319-b350-872e2bbb9836 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:05.456317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 157/333] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:49:05 2022] GET /identity/v3/users/ee92c2e937124bcca9b7fe6f61aed1b9 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:05.462428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3f9bc-380d-4ea9-a213-a800fba6d852 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:05.483780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:05.599695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3f9bc-380d-4ea9-a213-a800fba6d852 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:05.638753 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619138bd-264c-471f-bbbd-719a4602d70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:05.639661 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619138bd-264c-471f-bbbd-719a4602d70b None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:05.639947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619138bd-264c-471f-bbbd-719a4602d70b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:05.640257 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619138bd-264c-471f-bbbd-719a4602d70b None admin] PATH_INFO: `/v3/system/users/ee92c2e937124bcca9b7fe6f61aed1b9/roles/a2403691e5c440df989bc6aad7118272` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:05.641711 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619138bd-264c-471f-bbbd-719a4602d70b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ee92c2e937124bcca9b7fe6f61aed1b9, role_id=a2403691e5c440df989bc6aad7118272)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:05.643273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619138bd-264c-471f-bbbd-719a4602d70b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:05.717868 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 177/334] 10.222.0.69 () {58 vars in 1334 bytes} [Thu Jul 14 18:49:05 2022] PUT /identity/v3/system/users/ee92c2e937124bcca9b7fe6f61aed1b9/roles/a2403691e5c440df989bc6aad7118272 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 18:49:07.363414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05460221-652c-4c06-b7e0-7ba1794de374 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:07.364415 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05460221-652c-4c06-b7e0-7ba1794de374 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:07.364905 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05460221-652c-4c06-b7e0-7ba1794de374 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:07.366488 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 158/335] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:07.372090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce397376-47f4-448b-807c-52846bbcf88c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:07.372904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce397376-47f4-448b-807c-52846bbcf88c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:07.373398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce397376-47f4-448b-807c-52846bbcf88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:07.396156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ce397376-47f4-448b-807c-52846bbcf88c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:07.397679 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ce397376-47f4-448b-807c-52846bbcf88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:07.485256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 178/336] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:07.529963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05460221-652c-4c06-b7e0-7ba1794de374 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:07.533621 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:07.563618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa69d8-dd19-4aaf-9184-3c993b16108b None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:07.563930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa69d8-dd19-4aaf-9184-3c993b16108b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:07.564177 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa69d8-dd19-4aaf-9184-3c993b16108b None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:07.565558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 179/337] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:07.582390 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cd69e-e47b-4eb7-aaae-06da24f93a98 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:07.582675 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cd69e-e47b-4eb7-aaae-06da24f93a98 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:07.582961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cd69e-e47b-4eb7-aaae-06da24f93a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:07.612273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7c7cd69e-e47b-4eb7-aaae-06da24f93a98 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:07.614113 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7c7cd69e-e47b-4eb7-aaae-06da24f93a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:07.617929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:07.702858 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05460221-652c-4c06-b7e0-7ba1794de374 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:07.760068 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 180/338] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:49:07.767327 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fa543-c1b8-42ad-bfe6-1b373e45261e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:07.768131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fa543-c1b8-42ad-bfe6-1b373e45261e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:07.768416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fa543-c1b8-42ad-bfe6-1b373e45261e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:07.768695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fa543-c1b8-42ad-bfe6-1b373e45261e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:07.784351 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79fa543-c1b8-42ad-bfe6-1b373e45261e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:07.798218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b79fa543-c1b8-42ad-bfe6-1b373e45261e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:49:07.798218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:07.798218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:07.798218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:07.798218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:07.798218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:07.798218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:07.798218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:07.798218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:07.798218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:07.798218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:07.798218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:07.798218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:07.798218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:07.798218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:07.798218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:07.799101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:07.799101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:07.799101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:07.799101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:07.799101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:07.799101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:07.799101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:07.799101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:07.799101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:07.799101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:07.799101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:07.799101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:07.799101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:07.799101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:07.799101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:07.799101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:07.799925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:07.799925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:07.799925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:07.799925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:07.799925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:07.799925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:07.799925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:07.799925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:07.799925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:07.799925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:07.799925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:49:07.799925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:07.799925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 159/339] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:07.805492 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cd69e-e47b-4eb7-aaae-06da24f93a98 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:07.829850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fa543-c1b8-42ad-bfe6-1b373e45261e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:07.963656 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:07.987658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cd69e-e47b-4eb7-aaae-06da24f93a98 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:08.038086 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fa543-c1b8-42ad-bfe6-1b373e45261e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:08.056451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c49f30-efd9-47c4-a3b2-177f217329c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:08.057148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c49f30-efd9-47c4-a3b2-177f217329c4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:08.057435 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c49f30-efd9-47c4-a3b2-177f217329c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:08.057713 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c49f30-efd9-47c4-a3b2-177f217329c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:08.058476 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c49f30-efd9-47c4-a3b2-177f217329c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:08.058788 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c49f30-efd9-47c4-a3b2-177f217329c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:08.060224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c49f30-efd9-47c4-a3b2-177f217329c4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:08.067812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:08.075207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 181/340] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:49:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:08.080943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c49f30-efd9-47c4-a3b2-177f217329c4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:08.092875 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd42db-5a0f-41de-b00c-ed297537cbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:08.093708 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd42db-5a0f-41de-b00c-ed297537cbd2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:08.093998 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd42db-5a0f-41de-b00c-ed297537cbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:08.094280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd42db-5a0f-41de-b00c-ed297537cbd2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:08.120325 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd42db-5a0f-41de-b00c-ed297537cbd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:08.129901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-0efd42db-5a0f-41de-b00c-ed297537cbd2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:49:08.129901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:08.129901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:08.129901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:08.129901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:08.129901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:08.129901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:08.129901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:08.129901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:08.129901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:08.129901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:08.129901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:08.129901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:08.129901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:08.129901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:08.129901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:08.130797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:08.130797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:08.130797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:08.130797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:08.130797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:08.130797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:08.130797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:08.130797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:08.130797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:08.130797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:08.130797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:08.130797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:08.130797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:08.130797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:08.130797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:08.130797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:08.131614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:08.131614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:08.131614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:08.131614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:08.131614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:08.131614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:08.131614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:08.131614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:08.131614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:08.131614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:08.131614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:49:08.131614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:08.136320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 160/341] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:49:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:49:08.140902 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd42db-5a0f-41de-b00c-ed297537cbd2 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:08.290467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c49f30-efd9-47c4-a3b2-177f217329c4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:08.336507 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd42db-5a0f-41de-b00c-ed297537cbd2 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:08.387921 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704ca27-a95c-4d3f-9665-e3c735682d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:08.388607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704ca27-a95c-4d3f-9665-e3c735682d31 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:08.388901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704ca27-a95c-4d3f-9665-e3c735682d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:08.389182 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704ca27-a95c-4d3f-9665-e3c735682d31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:08.408750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:08.439914 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880a850-23df-495d-82ce-a8cc7abe6f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:08.440620 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880a850-23df-495d-82ce-a8cc7abe6f0b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:08.440910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880a850-23df-495d-82ce-a8cc7abe6f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:08.441190 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880a850-23df-495d-82ce-a8cc7abe6f0b None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:08.442602 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880a850-23df-495d-82ce-a8cc7abe6f0b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:08.442915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880a850-23df-495d-82ce-a8cc7abe6f0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:08.444382 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880a850-23df-495d-82ce-a8cc7abe6f0b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:08.470784 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 161/342] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:08.480291 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880a850-23df-495d-82ce-a8cc7abe6f0b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:08.504875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:08.513429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704ca27-a95c-4d3f-9665-e3c735682d31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:08.514953 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704ca27-a95c-4d3f-9665-e3c735682d31 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:08.696892 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880a850-23df-495d-82ce-a8cc7abe6f0b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:08.749252 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2dde5-62ac-41f2-b6f1-4f81673d7378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:08.750270 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2dde5-62ac-41f2-b6f1-4f81673d7378 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:08.750560 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 182/343] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 670 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 18:49:08.750984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2dde5-62ac-41f2-b6f1-4f81673d7378 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:08.751481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2dde5-62ac-41f2-b6f1-4f81673d7378 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:08.751965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b02e8-b22c-4a78-aabc-106361d7316d None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:08.752264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b02e8-b22c-4a78-aabc-106361d7316d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:08.752548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b02e8-b22c-4a78-aabc-106361d7316d None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:08.753870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 183/344] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:08.759290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08812b-5665-43b3-ab6a-aca35563e478 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:08.759581 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08812b-5665-43b3-ab6a-aca35563e478 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:08.759863 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08812b-5665-43b3-ab6a-aca35563e478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:08.790979 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0d08812b-5665-43b3-ab6a-aca35563e478 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:08.792240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0d08812b-5665-43b3-ab6a-aca35563e478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:08.831329 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:08.867396 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2dde5-62ac-41f2-b6f1-4f81673d7378 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:08.868937 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2dde5-62ac-41f2-b6f1-4f81673d7378 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:08.880027 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 184/345] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:49:08.881222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08812b-5665-43b3-ab6a-aca35563e478 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:08.921705 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:09.016388 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08812b-5665-43b3-ab6a-aca35563e478 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:09.057099 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a7d63-3af3-44ff-8a20-e0c99346605f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:09.058320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a7d63-3af3-44ff-8a20-e0c99346605f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:09.058829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a7d63-3af3-44ff-8a20-e0c99346605f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:09.059333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a7d63-3af3-44ff-8a20-e0c99346605f None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:09.060898 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a7d63-3af3-44ff-8a20-e0c99346605f None admin] RBAC: Authorizing `identity:get_project(project_id=b82ee754849c4779b408e856c429ab9b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:09.062681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a7d63-3af3-44ff-8a20-e0c99346605f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:09.064969 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 185/346] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:49:08 2022] GET /identity/v3/projects/b82ee754849c4779b408e856c429ab9b => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:09.065978 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a7d63-3af3-44ff-8a20-e0c99346605f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:09.078839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 162/347] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:09.083063 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe9cfc-39b1-4aed-be93-325024a2a7dc None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:09.084969 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe9cfc-39b1-4aed-be93-325024a2a7dc None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:09.084969 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe9cfc-39b1-4aed-be93-325024a2a7dc None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:09.086309 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 163/348] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:09.086967 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe9cfc-39b1-4aed-be93-325024a2a7dc None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:09.172175 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:09.176226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a7d63-3af3-44ff-8a20-e0c99346605f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:09.201701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:09.213948 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe9cfc-39b1-4aed-be93-325024a2a7dc None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:09.215932 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c664c2-acf3-4ace-9f0e-4789f8fcb675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:09.216951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c664c2-acf3-4ace-9f0e-4789f8fcb675 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:09.217420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c664c2-acf3-4ace-9f0e-4789f8fcb675 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:09.217875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c664c2-acf3-4ace-9f0e-4789f8fcb675 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:09.233283 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c664c2-acf3-4ace-9f0e-4789f8fcb675 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:09.242842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-89c664c2-acf3-4ace-9f0e-4789f8fcb675 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:49:09.242842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:09.242842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:09.242842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:09.242842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:09.242842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:09.242842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:09.242842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:09.242842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:09.242842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:09.242842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:09.242842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:09.242842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:09.242842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:09.242842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:09.242842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:09.243887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:09.243887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:09.243887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:09.243887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:09.243887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:09.243887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:09.243887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:09.243887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:09.243887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:09.243887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:09.243887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:09.243887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:09.243887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:09.243887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:09.243887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:09.243887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:09.244812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:09.244812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:09.244812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:09.244812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:09.244812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:09.244812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:09.244812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:09.244812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:09.244812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:09.244812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:09.244812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:49:09.244812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:09.246828 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 186/349] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:09.247777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c664c2-acf3-4ace-9f0e-4789f8fcb675 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:09.253723 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652ef0f-1150-448a-8578-4009cce095e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:09.254778 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652ef0f-1150-448a-8578-4009cce095e1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:09.255398 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652ef0f-1150-448a-8578-4009cce095e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:09.255699 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652ef0f-1150-448a-8578-4009cce095e1 None admin] PATH_INFO: `/v3/groups/c93c0d034ea446718a1169bb53fc913d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:09.262544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652ef0f-1150-448a-8578-4009cce095e1 None admin] RBAC: Authorizing `identity:get_group(group_id=c93c0d034ea446718a1169bb53fc913d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:09.264245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652ef0f-1150-448a-8578-4009cce095e1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:09.266217 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 164/350] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 14 18:49:09 2022] GET /identity/v3/groups/c93c0d034ea446718a1169bb53fc913d => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:09.267584 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53a6b2-a036-4ccc-8d3e-2e9937ada861 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:09.267877 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53a6b2-a036-4ccc-8d3e-2e9937ada861 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:09.268158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53a6b2-a036-4ccc-8d3e-2e9937ada861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:09.289647 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-8f53a6b2-a036-4ccc-8d3e-2e9937ada861 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:09.290852 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8f53a6b2-a036-4ccc-8d3e-2e9937ada861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:09.361009 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c664c2-acf3-4ace-9f0e-4789f8fcb675 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:09.375481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 165/351] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:09.377089 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f24f8-36fc-4a66-9499-9e2375727a42 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:09.377556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f24f8-36fc-4a66-9499-9e2375727a42 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:09.378005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f24f8-36fc-4a66-9499-9e2375727a42 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:09.380340 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 166/352] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:09.381224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f24f8-36fc-4a66-9499-9e2375727a42 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:09.413988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a312eb-8530-476a-bbbe-f9a15171e8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:09.414856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a312eb-8530-476a-bbbe-f9a15171e8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:09.415362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a312eb-8530-476a-bbbe-f9a15171e8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:09.415714 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a312eb-8530-476a-bbbe-f9a15171e8d7 None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:09.416854 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a312eb-8530-476a-bbbe-f9a15171e8d7 None admin] RBAC: Authorizing `identity:get_project(project_id=b82ee754849c4779b408e856c429ab9b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:09.418306 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a312eb-8530-476a-bbbe-f9a15171e8d7 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:09.420581 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 187/353] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:49:09 2022] GET /identity/v3/projects/b82ee754849c4779b408e856c429ab9b => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:09.420821 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a312eb-8530-476a-bbbe-f9a15171e8d7 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:09.458589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:09.488514 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:09.522447 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f24f8-36fc-4a66-9499-9e2375727a42 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:09.548411 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a312eb-8530-476a-bbbe-f9a15171e8d7 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:09.569080 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919af2a-4ee6-417e-9456-0d03af5bd30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:09.570092 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919af2a-4ee6-417e-9456-0d03af5bd30a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:09.570585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919af2a-4ee6-417e-9456-0d03af5bd30a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:09.570929 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919af2a-4ee6-417e-9456-0d03af5bd30a None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:09.571740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919af2a-4ee6-417e-9456-0d03af5bd30a None admin] RBAC: Adding query filter params (name=member) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:09.572054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919af2a-4ee6-417e-9456-0d03af5bd30a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:09.573444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919af2a-4ee6-417e-9456-0d03af5bd30a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:09.583913 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 167/354] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:49:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:09.589103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b9a82-627c-4dba-bc87-7890939bb63a None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:09.589103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b9a82-627c-4dba-bc87-7890939bb63a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:09.589103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b9a82-627c-4dba-bc87-7890939bb63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:09.600015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03720da3-87f3-400d-98d5-3952fdb60e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:09.600695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03720da3-87f3-400d-98d5-3952fdb60e7b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:09.601056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03720da3-87f3-400d-98d5-3952fdb60e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:09.601376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03720da3-87f3-400d-98d5-3952fdb60e7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:09.602175 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03720da3-87f3-400d-98d5-3952fdb60e7b None admin] RBAC: Adding query filter params (group.id=c93c0d034ea446718a1169bb53fc913d, role.id=ffed4493824a4040977d2cd1d057e07b, scope.project.id=b82ee754849c4779b408e856c429ab9b) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:09.602487 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03720da3-87f3-400d-98d5-3952fdb60e7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:09.603981 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03720da3-87f3-400d-98d5-3952fdb60e7b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:09.605596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f00b9a82-627c-4dba-bc87-7890939bb63a None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:09.606777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f00b9a82-627c-4dba-bc87-7890939bb63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:09.610101 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 188/355] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 14 18:49:09 2022] GET /identity/v3/role_assignments?group.id=c93c0d034ea446718a1169bb53fc913d&scope.project.id=b82ee754849c4779b408e856c429ab9b&role.id=ffed4493824a4040977d2cd1d057e07b => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:09.610751 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03720da3-87f3-400d-98d5-3952fdb60e7b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:09.706320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 168/356] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:09.707095 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b9a82-627c-4dba-bc87-7890939bb63a None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:09.756081 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:09.771018 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03720da3-87f3-400d-98d5-3952fdb60e7b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:09.819794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:09.826648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48ac03-1671-4b9d-b79e-fe3d93c224a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:09.827476 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48ac03-1671-4b9d-b79e-fe3d93c224a0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:09.827766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48ac03-1671-4b9d-b79e-fe3d93c224a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:09.828045 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48ac03-1671-4b9d-b79e-fe3d93c224a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:09.840328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48ac03-1671-4b9d-b79e-fe3d93c224a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:09.848318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-7b48ac03-1671-4b9d-b79e-fe3d93c224a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 18:49:09.848318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:09.848318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:09.848318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:09.848318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:09.848318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:09.848318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:09.848318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:09.848318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:09.848318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:09.848318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:09.848318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:09.848318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:09.848318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:09.848318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:09.848318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:09.849239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:09.849239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:09.849239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:09.849239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:09.849239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:09.849239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:09.849239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:09.849239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:09.849239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:09.849239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:09.849239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:09.849239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:09.849239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:09.849239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:09.849239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:09.849239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:09.850010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:09.850010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:09.850010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:09.850010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:09.850010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:09.850010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:09.850010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:09.850010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:09.850010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:09.850010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:09.850010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 18:49:09.850010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:09.850591 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 189/357] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:09.851925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48ac03-1671-4b9d-b79e-fe3d93c224a0 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:09.859113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b9a82-627c-4dba-bc87-7890939bb63a None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:09.905378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863c380-d053-42de-bb61-150cf6c47667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:09.907032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863c380-d053-42de-bb61-150cf6c47667 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:09.907769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863c380-d053-42de-bb61-150cf6c47667 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:09.908226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863c380-d053-42de-bb61-150cf6c47667 None admin] PATH_INFO: `/v3/groups/c93c0d034ea446718a1169bb53fc913d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:09.910281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5863c380-d053-42de-bb61-150cf6c47667 None admin] RBAC: Authorizing `identity:get_group(group_id=c93c0d034ea446718a1169bb53fc913d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:09.911968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5863c380-d053-42de-bb61-150cf6c47667 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:09.914970 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 169/358] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 14 18:49:09 2022] GET /identity/v3/groups/c93c0d034ea446718a1169bb53fc913d => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:09.915980 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863c380-d053-42de-bb61-150cf6c47667 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:09.973758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48ac03-1671-4b9d-b79e-fe3d93c224a0 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:10.025875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d30514-ef90-4e17-8afb-e2abca7de903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:10.026886 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d30514-ef90-4e17-8afb-e2abca7de903 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:10.027191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d30514-ef90-4e17-8afb-e2abca7de903 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:10.027486 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d30514-ef90-4e17-8afb-e2abca7de903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:10.045893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863c380-d053-42de-bb61-150cf6c47667 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:10.052398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:10.100113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7a222-9c33-47e0-a0e4-28db5a58591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:10.100113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7a222-9c33-47e0-a0e4-28db5a58591e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:10.100113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7a222-9c33-47e0-a0e4-28db5a58591e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:10.100113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7a222-9c33-47e0-a0e4-28db5a58591e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:10.110075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:10.115491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d30514-ef90-4e17-8afb-e2abca7de903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:10.117116 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7a222-9c33-47e0-a0e4-28db5a58591e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:10.117917 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d30514-ef90-4e17-8afb-e2abca7de903 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:10.125414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-75d7a222-9c33-47e0-a0e4-28db5a58591e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:49:10.125414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:10.125414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:10.125414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:10.125414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:10.125414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:10.125414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:10.125414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:10.125414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:10.125414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:10.125414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:10.125414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:10.125414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:10.125414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:10.125414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:10.125414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:10.126277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:10.126277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:10.126277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:10.126277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:10.126277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:10.126277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:10.126277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:10.126277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:10.126277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:10.126277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:10.126277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:10.126277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:10.126277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:10.126277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:10.126277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:10.126277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:10.127073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:10.127073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:10.127073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:10.127073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:10.127073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:10.127073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:10.127073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:10.127073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:10.127073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:10.127073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:10.127073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:49:10.127073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:10.131990 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 170/359] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:49:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:10.132530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7a222-9c33-47e0-a0e4-28db5a58591e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:10.249536 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 190/360] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:10.251026 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d30514-ef90-4e17-8afb-e2abca7de903 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:10.260899 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7a222-9c33-47e0-a0e4-28db5a58591e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:10.307001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd921f-6bc5-43be-a7bc-8a103b6560ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:10.307714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd921f-6bc5-43be-a7bc-8a103b6560ee None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:10.307997 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd921f-6bc5-43be-a7bc-8a103b6560ee None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:10.308275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd921f-6bc5-43be-a7bc-8a103b6560ee None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:10.309027 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd921f-6bc5-43be-a7bc-8a103b6560ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:10.309339 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd921f-6bc5-43be-a7bc-8a103b6560ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:10.310723 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd921f-6bc5-43be-a7bc-8a103b6560ee None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:10.320535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 171/361] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:49:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:10.321646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f7494-bfbb-4d01-8bef-488df3a49837 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:10.321934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f7494-bfbb-4d01-8bef-488df3a49837 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:10.322249 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f7494-bfbb-4d01-8bef-488df3a49837 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:10.323646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 172/362] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:10.324186 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f7494-bfbb-4d01-8bef-488df3a49837 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:10.328885 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:10.395455 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d30514-ef90-4e17-8afb-e2abca7de903 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:10.402739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:10.442593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb1ce3-d620-4e79-b79f-8953a4929855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:10.443517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb1ce3-d620-4e79-b79f-8953a4929855 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:10.443835 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb1ce3-d620-4e79-b79f-8953a4929855 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:10.444120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb1ce3-d620-4e79-b79f-8953a4929855 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:10.444957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb1ce3-d620-4e79-b79f-8953a4929855 None admin] RBAC: Adding query filter params (group.id=c93c0d034ea446718a1169bb53fc913d, role.id=394e165201dc41c8b3255de480821b1f, scope.project.id=b82ee754849c4779b408e856c429ab9b) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:10.445352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb1ce3-d620-4e79-b79f-8953a4929855 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:10.446721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb1ce3-d620-4e79-b79f-8953a4929855 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:10.456867 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 191/363] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 14 18:49:10 2022] GET /identity/v3/role_assignments?group.id=c93c0d034ea446718a1169bb53fc913d&scope.project.id=b82ee754849c4779b408e856c429ab9b&role.id=394e165201dc41c8b3255de480821b1f => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:10.461819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb1ce3-d620-4e79-b79f-8953a4929855 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:10.463079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f7494-bfbb-4d01-8bef-488df3a49837 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:10.522361 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0690aca6-e704-4a5b-9bc1-b11711a841de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:10.523185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690aca6-e704-4a5b-9bc1-b11711a841de None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:10.523478 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690aca6-e704-4a5b-9bc1-b11711a841de None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:10.523804 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690aca6-e704-4a5b-9bc1-b11711a841de None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:10.524593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0690aca6-e704-4a5b-9bc1-b11711a841de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:10.524906 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0690aca6-e704-4a5b-9bc1-b11711a841de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:10.526656 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0690aca6-e704-4a5b-9bc1-b11711a841de None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:10.537431 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 173/364] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:10.538317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0690aca6-e704-4a5b-9bc1-b11711a841de None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:10.602045 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb1ce3-d620-4e79-b79f-8953a4929855 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:10.619370 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:10.647696 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96858e33-9111-4a9c-acf0-e119eeb5bfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:10.648350 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96858e33-9111-4a9c-acf0-e119eeb5bfa7 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:10.648644 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96858e33-9111-4a9c-acf0-e119eeb5bfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:10.648925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96858e33-9111-4a9c-acf0-e119eeb5bfa7 None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:10.650145 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96858e33-9111-4a9c-acf0-e119eeb5bfa7 None admin] RBAC: Authorizing `identity:get_project(project_id=45bac146c9d842bf82503d3f2b145128)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:10.651925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96858e33-9111-4a9c-acf0-e119eeb5bfa7 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:10.653945 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 192/365] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:49:10 2022] GET /identity/v3/projects/45bac146c9d842bf82503d3f2b145128 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:10.656613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bce3c-1ad6-4f69-85d7-79872a861859 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:10.656974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bce3c-1ad6-4f69-85d7-79872a861859 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:10.657272 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bce3c-1ad6-4f69-85d7-79872a861859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:10.672246 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0690aca6-e704-4a5b-9bc1-b11711a841de None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:10.680838 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-f99bce3c-1ad6-4f69-85d7-79872a861859 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:10.682160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f99bce3c-1ad6-4f69-85d7-79872a861859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:10.696674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:10.710125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821137a-8dff-41d9-908e-57db260b3f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:10.711322 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821137a-8dff-41d9-908e-57db260b3f38 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:10.711670 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821137a-8dff-41d9-908e-57db260b3f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:10.712077 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821137a-8dff-41d9-908e-57db260b3f38 None admin] PATH_INFO: `/v3/users/ee92c2e937124bcca9b7fe6f61aed1b9` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:10.713400 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821137a-8dff-41d9-908e-57db260b3f38 None admin] RBAC: Authorizing `identity:get_user(user_id=ee92c2e937124bcca9b7fe6f61aed1b9)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:10.714986 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821137a-8dff-41d9-908e-57db260b3f38 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:10.716800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 174/366] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:49:10 2022] GET /identity/v3/users/ee92c2e937124bcca9b7fe6f61aed1b9 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:10.717697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821137a-8dff-41d9-908e-57db260b3f38 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:10.768588 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 193/367] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:10.770216 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99bce3c-1ad6-4f69-85d7-79872a861859 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:10.835914 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821137a-8dff-41d9-908e-57db260b3f38 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:10.876650 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ac26f-f66b-47b4-adfb-4c4692204042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:10.877296 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ac26f-f66b-47b4-adfb-4c4692204042 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:10.879508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ac26f-f66b-47b4-adfb-4c4692204042 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:10.879943 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ac26f-f66b-47b4-adfb-4c4692204042 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:10.921838 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99bce3c-1ad6-4f69-85d7-79872a861859 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:10.942927 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:10.951827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:10.966963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ac26f-f66b-47b4-adfb-4c4692204042 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:10.968493 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ac26f-f66b-47b4-adfb-4c4692204042 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:10.973720 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa6e54-fcf6-4269-9bca-509f18749957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:10.974585 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa6e54-fcf6-4269-9bca-509f18749957 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:10.974875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa6e54-fcf6-4269-9bca-509f18749957 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:10.975156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa6e54-fcf6-4269-9bca-509f18749957 None admin] PATH_INFO: `/v3/groups/c93c0d034ea446718a1169bb53fc913d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:10.976413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa6e54-fcf6-4269-9bca-509f18749957 None admin] RBAC: Authorizing `identity:get_group(group_id=c93c0d034ea446718a1169bb53fc913d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:10.978944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa6e54-fcf6-4269-9bca-509f18749957 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:10.981272 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 194/368] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 14 18:49:10 2022] GET /identity/v3/groups/c93c0d034ea446718a1169bb53fc913d => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:10.982372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa6e54-fcf6-4269-9bca-509f18749957 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:11.112374 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 175/369] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:11.113264 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ac26f-f66b-47b4-adfb-4c4692204042 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:11.113917 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa6e54-fcf6-4269-9bca-509f18749957 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:11.157853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e79bc-bfc5-4a94-b3fb-bfcae0ac2e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:11.158522 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e79bc-bfc5-4a94-b3fb-bfcae0ac2e2c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:11.159059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e79bc-bfc5-4a94-b3fb-bfcae0ac2e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:11.159565 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e79bc-bfc5-4a94-b3fb-bfcae0ac2e2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:11.160715 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e79bc-bfc5-4a94-b3fb-bfcae0ac2e2c None admin] RBAC: Adding query filter params (role.id=a2403691e5c440df989bc6aad7118272, user.id=ee92c2e937124bcca9b7fe6f61aed1b9, scope.system=all) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:11.161214 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e79bc-bfc5-4a94-b3fb-bfcae0ac2e2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:11.162977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e79bc-bfc5-4a94-b3fb-bfcae0ac2e2c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:11.180397 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 195/370] 10.222.0.69 () {56 vars in 1308 bytes} [Thu Jul 14 18:49:10 2022] GET /identity/v3/role_assignments?user.id=ee92c2e937124bcca9b7fe6f61aed1b9&scope.system=all&role.id=a2403691e5c440df989bc6aad7118272 => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:11.181684 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e79bc-bfc5-4a94-b3fb-bfcae0ac2e2c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:11.235327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:11.270357 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:11.303657 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e79bc-bfc5-4a94-b3fb-bfcae0ac2e2c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:11.305544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ac26f-f66b-47b4-adfb-4c4692204042 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:11.342654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537ea35-33fd-4fd3-ad9c-7fc6d2868d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:11.343411 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537ea35-33fd-4fd3-ad9c-7fc6d2868d8f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:11.343702 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537ea35-33fd-4fd3-ad9c-7fc6d2868d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:11.344027 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537ea35-33fd-4fd3-ad9c-7fc6d2868d8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:11.344875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537ea35-33fd-4fd3-ad9c-7fc6d2868d8f None admin] RBAC: Adding query filter params (group.id=c93c0d034ea446718a1169bb53fc913d, role.id=ffed4493824a4040977d2cd1d057e07b, scope.project.id=45bac146c9d842bf82503d3f2b145128) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:11.345193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537ea35-33fd-4fd3-ad9c-7fc6d2868d8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:11.346716 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537ea35-33fd-4fd3-ad9c-7fc6d2868d8f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:11.353684 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 196/371] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 14 18:49:11 2022] GET /identity/v3/role_assignments?group.id=c93c0d034ea446718a1169bb53fc913d&scope.project.id=45bac146c9d842bf82503d3f2b145128&role.id=ffed4493824a4040977d2cd1d057e07b => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:11.354622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537ea35-33fd-4fd3-ad9c-7fc6d2868d8f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:11.385577 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c82a14-a58e-4d1b-84f6-aa0b22f2e618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:11.386780 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c82a14-a58e-4d1b-84f6-aa0b22f2e618 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:11.387310 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c82a14-a58e-4d1b-84f6-aa0b22f2e618 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:11.387800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c82a14-a58e-4d1b-84f6-aa0b22f2e618 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:11.416784 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c82a14-a58e-4d1b-84f6-aa0b22f2e618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:11.425481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-66c82a14-a58e-4d1b-84f6-aa0b22f2e618 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:49:11.425481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:11.425481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:11.425481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:11.425481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:11.425481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:11.425481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:11.425481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:11.425481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:11.425481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:11.425481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:11.425481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:11.425481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:11.425481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:11.425481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:11.425481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:11.435279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:11.435279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:11.435279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:11.435279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:11.435279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:11.435279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:11.435279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:11.435279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:11.435279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:11.435279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:11.435279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:11.435279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:11.435279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:11.435279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:11.435279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:11.435279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:11.436079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:11.436079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:11.436079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:11.436079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:11.436079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:11.436079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:11.436079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:11.436079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:11.436079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:11.436079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:11.436079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:49:11.436079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:11.436079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 176/372] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:49:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:11.437378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c82a14-a58e-4d1b-84f6-aa0b22f2e618 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:11.473670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537ea35-33fd-4fd3-ad9c-7fc6d2868d8f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:11.475389 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:11.513005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454aa25-a2d4-4d15-bc74-a00d9a9d3f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:11.513964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454aa25-a2d4-4d15-bc74-a00d9a9d3f93 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:11.514253 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454aa25-a2d4-4d15-bc74-a00d9a9d3f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:11.514578 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454aa25-a2d4-4d15-bc74-a00d9a9d3f93 None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:11.515864 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454aa25-a2d4-4d15-bc74-a00d9a9d3f93 None admin] RBAC: Authorizing `identity:get_project(project_id=45bac146c9d842bf82503d3f2b145128)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:11.518214 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454aa25-a2d4-4d15-bc74-a00d9a9d3f93 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:11.520572 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 197/373] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:49:11 2022] GET /identity/v3/projects/45bac146c9d842bf82503d3f2b145128 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:11.527408 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454aa25-a2d4-4d15-bc74-a00d9a9d3f93 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:11.605491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c82a14-a58e-4d1b-84f6-aa0b22f2e618 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:11.661668 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c894a4-d8d2-47d4-8ac8-fff0ee14eadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:11.662460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c894a4-d8d2-47d4-8ac8-fff0ee14eadf None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:11.662825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c894a4-d8d2-47d4-8ac8-fff0ee14eadf None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:11.663205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c894a4-d8d2-47d4-8ac8-fff0ee14eadf None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:11.664055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c894a4-d8d2-47d4-8ac8-fff0ee14eadf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:11.664370 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c894a4-d8d2-47d4-8ac8-fff0ee14eadf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:11.666457 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c894a4-d8d2-47d4-8ac8-fff0ee14eadf None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:11.676903 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 177/374] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:11.680898 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454aa25-a2d4-4d15-bc74-a00d9a9d3f93 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:11.682623 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c894a4-d8d2-47d4-8ac8-fff0ee14eadf None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:11.705849 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:11.732335 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d31bb-6002-42ed-961f-bc8da2760c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:11.733524 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d31bb-6002-42ed-961f-bc8da2760c4e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:11.735183 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d31bb-6002-42ed-961f-bc8da2760c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:11.735648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d31bb-6002-42ed-961f-bc8da2760c4e None admin] PATH_INFO: `/v3/groups/c93c0d034ea446718a1169bb53fc913d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:11.737883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d31bb-6002-42ed-961f-bc8da2760c4e None admin] RBAC: Authorizing `identity:get_group(group_id=c93c0d034ea446718a1169bb53fc913d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:11.740823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d31bb-6002-42ed-961f-bc8da2760c4e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:11.742882 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 198/375] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 14 18:49:11 2022] GET /identity/v3/groups/c93c0d034ea446718a1169bb53fc913d => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:11.751913 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d31bb-6002-42ed-961f-bc8da2760c4e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:11.814895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:11.863997 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c894a4-d8d2-47d4-8ac8-fff0ee14eadf None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:11.924255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d31bb-6002-42ed-961f-bc8da2760c4e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:11.930992 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b4f5d-4749-47c1-87d5-05679b09e870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:11.932003 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b4f5d-4749-47c1-87d5-05679b09e870 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:11.932472 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b4f5d-4749-47c1-87d5-05679b09e870 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:11.932928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b4f5d-4749-47c1-87d5-05679b09e870 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:11.976518 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd139445-8389-426c-9203-0a2c316ef69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:11.977456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd139445-8389-426c-9203-0a2c316ef69c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:11.977920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd139445-8389-426c-9203-0a2c316ef69c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:11.978371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd139445-8389-426c-9203-0a2c316ef69c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:11.979454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd139445-8389-426c-9203-0a2c316ef69c None admin] RBAC: Adding query filter params (group.id=c93c0d034ea446718a1169bb53fc913d, role.id=394e165201dc41c8b3255de480821b1f, scope.project.id=45bac146c9d842bf82503d3f2b145128) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:11.979965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd139445-8389-426c-9203-0a2c316ef69c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:11.981944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd139445-8389-426c-9203-0a2c316ef69c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:11.989244 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 199/376] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 14 18:49:11 2022] GET /identity/v3/role_assignments?group.id=c93c0d034ea446718a1169bb53fc913d&scope.project.id=45bac146c9d842bf82503d3f2b145128&role.id=394e165201dc41c8b3255de480821b1f => generated 260 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:12.041876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b4f5d-4749-47c1-87d5-05679b09e870 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:12.043425 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b4f5d-4749-47c1-87d5-05679b09e870 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:12.059575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:12.152063 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 178/377] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:12.161234 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd139445-8389-426c-9203-0a2c316ef69c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:12.162231 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbcf97-2483-497a-a7ae-df06e0ae1959 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:12.162528 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbcf97-2483-497a-a7ae-df06e0ae1959 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:12.162846 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbcf97-2483-497a-a7ae-df06e0ae1959 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:12.164358 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 179/378] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:49:12.172901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1751e7-8399-446f-a387-3ba9b3b05274 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:12.173188 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1751e7-8399-446f-a387-3ba9b3b05274 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:12.173543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1751e7-8399-446f-a387-3ba9b3b05274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:12.195892 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7c1751e7-8399-446f-a387-3ba9b3b05274 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:12.197230 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7c1751e7-8399-446f-a387-3ba9b3b05274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:12.288120 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 180/379] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:49:12.311603 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1751e7-8399-446f-a387-3ba9b3b05274 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:12.337156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd139445-8389-426c-9203-0a2c316ef69c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:12.338794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:12.359928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:12.377711 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106deb6-3777-45ff-acb2-f14d3ff1c02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:12.379464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106deb6-3777-45ff-acb2-f14d3ff1c02f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:12.381118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106deb6-3777-45ff-acb2-f14d3ff1c02f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:12.381602 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106deb6-3777-45ff-acb2-f14d3ff1c02f None admin] PATH_INFO: `/v3/projects/1ecc6a6cebab48d4b8cfff094ca5066b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:12.383046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106deb6-3777-45ff-acb2-f14d3ff1c02f None admin] RBAC: Authorizing `identity:get_project(project_id=1ecc6a6cebab48d4b8cfff094ca5066b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:12.384863 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106deb6-3777-45ff-acb2-f14d3ff1c02f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:12.387097 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 200/380] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:49:12 2022] GET /identity/v3/projects/1ecc6a6cebab48d4b8cfff094ca5066b => generated 344 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:12.392248 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106deb6-3777-45ff-acb2-f14d3ff1c02f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:12.422625 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1751e7-8399-446f-a387-3ba9b3b05274 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:12.473994 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa12548-b681-4ba4-bc4b-4f046a3c100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:12.475120 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa12548-b681-4ba4-bc4b-4f046a3c100b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:12.475613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa12548-b681-4ba4-bc4b-4f046a3c100b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:12.476069 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa12548-b681-4ba4-bc4b-4f046a3c100b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:12.490243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa12548-b681-4ba4-bc4b-4f046a3c100b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:12.499759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-8fa12548-b681-4ba4-bc4b-4f046a3c100b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:49:12.499759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:12.499759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:12.499759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:12.499759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:12.499759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:12.499759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:12.499759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:12.499759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:12.499759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:12.499759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:12.499759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:12.499759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:12.499759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:12.499759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:12.499759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:12.500667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:12.500667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:12.500667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:12.500667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:12.500667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:12.500667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:12.500667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:12.500667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:12.500667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:12.500667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:12.500667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:12.500667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:12.500667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:12.500667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:12.500667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:12.500667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:12.501460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:12.501460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:12.501460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:12.501460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:12.501460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:12.501460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:12.501460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:12.501460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:12.501460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:12.501460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:12.501460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:49:12.501460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:12.504675 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 181/381] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:49:12.510609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa12548-b681-4ba4-bc4b-4f046a3c100b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:12.535832 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106deb6-3777-45ff-acb2-f14d3ff1c02f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:12.585575 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f813c-e11a-42a4-afb3-74c712f59118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:12.586302 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f813c-e11a-42a4-afb3-74c712f59118 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:12.586598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f813c-e11a-42a4-afb3-74c712f59118 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:12.586878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f813c-e11a-42a4-afb3-74c712f59118 None admin] PATH_INFO: `/v3/groups/708197382e774537a8f99f8da9b48d99` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:12.595595 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f813c-e11a-42a4-afb3-74c712f59118 None admin] RBAC: Authorizing `identity:get_group(group_id=708197382e774537a8f99f8da9b48d99)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:12.598009 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f813c-e11a-42a4-afb3-74c712f59118 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:12.600329 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 201/382] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 14 18:49:12 2022] GET /identity/v3/groups/708197382e774537a8f99f8da9b48d99 => generated 229 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:12.605813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f813c-e11a-42a4-afb3-74c712f59118 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:12.652812 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa12548-b681-4ba4-bc4b-4f046a3c100b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:12.655272 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:12.676575 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:12.718029 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f791031-6794-4f32-93ff-1b6b2616ce01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:12.719003 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f791031-6794-4f32-93ff-1b6b2616ce01 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:12.719488 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f791031-6794-4f32-93ff-1b6b2616ce01 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:12.719944 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f791031-6794-4f32-93ff-1b6b2616ce01 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:12.720895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f791031-6794-4f32-93ff-1b6b2616ce01 None admin] RBAC: Adding query filter params (name=member) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:12.721384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f791031-6794-4f32-93ff-1b6b2616ce01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:12.723045 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f791031-6794-4f32-93ff-1b6b2616ce01 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:12.740102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 182/383] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:49:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:12.747222 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f791031-6794-4f32-93ff-1b6b2616ce01 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:12.802623 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f813c-e11a-42a4-afb3-74c712f59118 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:12.856055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df097b-5176-4291-b723-5b581108a332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:12.856759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df097b-5176-4291-b723-5b581108a332 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:12.857134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df097b-5176-4291-b723-5b581108a332 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:12.857488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df097b-5176-4291-b723-5b581108a332 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:12.858499 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df097b-5176-4291-b723-5b581108a332 None admin] RBAC: Adding query filter params (group.id=708197382e774537a8f99f8da9b48d99, role.id=27fc14631c494ca0b2dc7f708be016c2, scope.project.id=1ecc6a6cebab48d4b8cfff094ca5066b) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:12.858884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df097b-5176-4291-b723-5b581108a332 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:12.860566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df097b-5176-4291-b723-5b581108a332 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:12.868292 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 202/384] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 14 18:49:12 2022] GET /identity/v3/role_assignments?group.id=708197382e774537a8f99f8da9b48d99&scope.project.id=1ecc6a6cebab48d4b8cfff094ca5066b&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 260 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:12.897813 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f791031-6794-4f32-93ff-1b6b2616ce01 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:12.940320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bfd3d-07f8-4065-8aaf-c4431410aa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:12.942068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bfd3d-07f8-4065-8aaf-c4431410aa46 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:12.942537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bfd3d-07f8-4065-8aaf-c4431410aa46 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:12.943020 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bfd3d-07f8-4065-8aaf-c4431410aa46 None admin] PATH_INFO: `/v3/groups/c93c0d034ea446718a1169bb53fc913d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:12.945040 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62bfd3d-07f8-4065-8aaf-c4431410aa46 None admin] RBAC: Authorizing `identity:get_group(group_id=c93c0d034ea446718a1169bb53fc913d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:12.946738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62bfd3d-07f8-4065-8aaf-c4431410aa46 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:12.948927 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 183/385] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 14 18:49:12 2022] GET /identity/v3/groups/c93c0d034ea446718a1169bb53fc913d => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:12.954731 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df097b-5176-4291-b723-5b581108a332 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:13.078784 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:13.083860 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df097b-5176-4291-b723-5b581108a332 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:13.096800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2f10d-f3ef-4d16-b3d1-437c28fd175f None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:13.097280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2f10d-f3ef-4d16-b3d1-437c28fd175f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:13.098276 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2f10d-f3ef-4d16-b3d1-437c28fd175f None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:13.100083 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 184/386] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:13.105473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc223228-6620-4c67-8063-926ac706b321 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:13.105921 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc223228-6620-4c67-8063-926ac706b321 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:13.106364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc223228-6620-4c67-8063-926ac706b321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:13.124171 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa9234-f068-4884-a05c-7400a8a082de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:13.125047 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa9234-f068-4884-a05c-7400a8a082de None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:13.125333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa9234-f068-4884-a05c-7400a8a082de None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:13.125611 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa9234-f068-4884-a05c-7400a8a082de None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:13.132017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa9234-f068-4884-a05c-7400a8a082de None admin] RBAC: Authorizing `identity:get_project(project_id=b82ee754849c4779b408e856c429ab9b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:13.133896 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa9234-f068-4884-a05c-7400a8a082de None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:13.136026 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 203/387] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:49:12 2022] GET /identity/v3/projects/b82ee754849c4779b408e856c429ab9b => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:13.136138 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-fc223228-6620-4c67-8063-926ac706b321 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:13.137385 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-fc223228-6620-4c67-8063-926ac706b321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:13.151576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa9234-f068-4884-a05c-7400a8a082de None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:13.172929 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:13.265048 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa9234-f068-4884-a05c-7400a8a082de None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:13.281770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 185/388] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:13.303388 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964e5aa-6d2b-49bd-be4e-4fb065c5da0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:13.304290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964e5aa-6d2b-49bd-be4e-4fb065c5da0c None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:13.304576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964e5aa-6d2b-49bd-be4e-4fb065c5da0c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:13.304851 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964e5aa-6d2b-49bd-be4e-4fb065c5da0c None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b/groups/c93c0d034ea446718a1169bb53fc913d/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:13.306874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964e5aa-6d2b-49bd-be4e-4fb065c5da0c None admin] RBAC: Authorizing `identity:create_grant(project_id=b82ee754849c4779b408e856c429ab9b, group_id=c93c0d034ea446718a1169bb53fc913d, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:13.308345 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964e5aa-6d2b-49bd-be4e-4fb065c5da0c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:13.317934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc223228-6620-4c67-8063-926ac706b321 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:13.435502 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc223228-6620-4c67-8063-926ac706b321 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:13.445418 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 204/389] 10.222.0.69 () {58 vars in 1442 bytes} [Thu Jul 14 18:49:13 2022] PUT /identity/v3/projects/b82ee754849c4779b408e856c429ab9b/groups/c93c0d034ea446718a1169bb53fc913d/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 18:49:13.475746 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621eefd6-ba5d-4a28-84a7-58aaed2a7f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:13.476601 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eefd6-ba5d-4a28-84a7-58aaed2a7f47 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:13.476895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eefd6-ba5d-4a28-84a7-58aaed2a7f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:13.477178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eefd6-ba5d-4a28-84a7-58aaed2a7f47 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:13.489816 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621eefd6-ba5d-4a28-84a7-58aaed2a7f47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:13.498585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-621eefd6-ba5d-4a28-84a7-58aaed2a7f47 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:49:13.498585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:13.498585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:13.498585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:13.498585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:13.498585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:13.498585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:13.498585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:13.498585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:13.498585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:13.498585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:13.498585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:13.498585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:13.498585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:13.498585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:13.498585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:13.499612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:13.499612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:13.499612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:13.499612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:13.499612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:13.499612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:13.499612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:13.499612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:13.499612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:13.499612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:13.499612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:13.499612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:13.499612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:13.499612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:13.499612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:13.499612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:13.500394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:13.500394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:13.500394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:13.500394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:13.500394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:13.500394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:13.500394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:13.500394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:13.500394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:13.500394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:13.500394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:49:13.500394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:13.500394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 186/390] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:49:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:49:13.506461 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964e5aa-6d2b-49bd-be4e-4fb065c5da0c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:13.545362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:13.651900 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964e5aa-6d2b-49bd-be4e-4fb065c5da0c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:13.689674 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9822f55-39f2-4c3e-928b-bea26142f01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:13.690277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9822f55-39f2-4c3e-928b-bea26142f01e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:13.690565 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9822f55-39f2-4c3e-928b-bea26142f01e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:13.690844 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9822f55-39f2-4c3e-928b-bea26142f01e None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:13.691654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9822f55-39f2-4c3e-928b-bea26142f01e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:13.691961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9822f55-39f2-4c3e-928b-bea26142f01e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:13.693346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9822f55-39f2-4c3e-928b-bea26142f01e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:13.702699 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 205/391] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:13.708857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621eefd6-ba5d-4a28-84a7-58aaed2a7f47 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:13.741467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:13.844910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621eefd6-ba5d-4a28-84a7-58aaed2a7f47 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:13.898609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46950a73-3f8f-4d0a-a00e-5bd663fa78ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:13.899496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46950a73-3f8f-4d0a-a00e-5bd663fa78ff None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:13.899781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46950a73-3f8f-4d0a-a00e-5bd663fa78ff None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:13.900054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46950a73-3f8f-4d0a-a00e-5bd663fa78ff None admin] PATH_INFO: `/v3/groups/c93c0d034ea446718a1169bb53fc913d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:13.901501 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46950a73-3f8f-4d0a-a00e-5bd663fa78ff None admin] RBAC: Authorizing `identity:get_group(group_id=c93c0d034ea446718a1169bb53fc913d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:13.902955 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46950a73-3f8f-4d0a-a00e-5bd663fa78ff None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:13.904614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 187/392] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 14 18:49:13 2022] GET /identity/v3/groups/c93c0d034ea446718a1169bb53fc913d => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:13.910272 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9822f55-39f2-4c3e-928b-bea26142f01e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:14.042473 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:14.054211 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9822f55-39f2-4c3e-928b-bea26142f01e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:14.092914 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa614ff-af01-44f5-b468-b5eb52ff5089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:14.093785 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa614ff-af01-44f5-b468-b5eb52ff5089 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:14.094068 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa614ff-af01-44f5-b468-b5eb52ff5089 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:14.094342 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa614ff-af01-44f5-b468-b5eb52ff5089 None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:14.095556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa614ff-af01-44f5-b468-b5eb52ff5089 None admin] RBAC: Authorizing `identity:get_project(project_id=b82ee754849c4779b408e856c429ab9b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:14.097004 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa614ff-af01-44f5-b468-b5eb52ff5089 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:14.098734 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 206/393] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:49:13 2022] GET /identity/v3/projects/b82ee754849c4779b408e856c429ab9b => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:14.104591 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46950a73-3f8f-4d0a-a00e-5bd663fa78ff None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:14.113014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d1976-72c2-438d-a891-09591dd9fc30 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:14.113648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d1976-72c2-438d-a891-09591dd9fc30 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:14.114124 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d1976-72c2-438d-a891-09591dd9fc30 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:14.115983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 207/394] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:49:14.121162 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbf998-9fd8-4504-a4cb-3c0fada97c51 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:14.121683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbf998-9fd8-4504-a4cb-3c0fada97c51 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:14.122173 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbf998-9fd8-4504-a4cb-3c0fada97c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:14.146095 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d8fbf998-9fd8-4504-a4cb-3c0fada97c51 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:14.149988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d8fbf998-9fd8-4504-a4cb-3c0fada97c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:14.214124 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:14.231450 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46950a73-3f8f-4d0a-a00e-5bd663fa78ff None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:14.238628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 208/395] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:49:14.262281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbf998-9fd8-4504-a4cb-3c0fada97c51 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:14.324240 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5c11f-875f-4acd-906b-e10c9fc238fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:14.325521 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5c11f-875f-4acd-906b-e10c9fc238fc None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:14.326001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5c11f-875f-4acd-906b-e10c9fc238fc None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:14.326470 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5c11f-875f-4acd-906b-e10c9fc238fc None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b/groups/c93c0d034ea446718a1169bb53fc913d/roles/394e165201dc41c8b3255de480821b1f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:14.329043 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5c11f-875f-4acd-906b-e10c9fc238fc None admin] RBAC: Authorizing `identity:create_grant(project_id=b82ee754849c4779b408e856c429ab9b, group_id=c93c0d034ea446718a1169bb53fc913d, role_id=394e165201dc41c8b3255de480821b1f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:14.330732 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5c11f-875f-4acd-906b-e10c9fc238fc None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:14.372356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:14.383347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbf998-9fd8-4504-a4cb-3c0fada97c51 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:14.420509 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9f7ec-6b59-4f41-bf06-b4f8d84bd5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:14.421429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9f7ec-6b59-4f41-bf06-b4f8d84bd5fa None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:14.421723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9f7ec-6b59-4f41-bf06-b4f8d84bd5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:14.422005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9f7ec-6b59-4f41-bf06-b4f8d84bd5fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:14.434730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9f7ec-6b59-4f41-bf06-b4f8d84bd5fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:14.438635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 188/396] 10.222.0.69 () {58 vars in 1442 bytes} [Thu Jul 14 18:49:14 2022] PUT /identity/v3/projects/b82ee754849c4779b408e856c429ab9b/groups/c93c0d034ea446718a1169bb53fc913d/roles/394e165201dc41c8b3255de480821b1f => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:49:14.443843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-fab9f7ec-6b59-4f41-bf06-b4f8d84bd5fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:49:14.443843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:14.443843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:14.443843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:14.443843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:14.443843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:14.443843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:14.443843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:14.443843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:14.443843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:14.443843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:14.443843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:14.443843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:14.443843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:14.443843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:14.443843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:14.444757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:14.444757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:14.444757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:14.444757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:14.444757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:14.444757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:14.444757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:14.444757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:14.444757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:14.444757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:14.444757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:14.444757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:14.444757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:14.444757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:14.444757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:14.444757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:14.446220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:14.446220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:14.446220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:14.446220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:14.446220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:14.446220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:14.446220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:14.446220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:14.446220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:14.446220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:14.446220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:49:14.446220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:14.446220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 209/397] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:49:14.452345 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5c11f-875f-4acd-906b-e10c9fc238fc None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:14.604817 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5c11f-875f-4acd-906b-e10c9fc238fc None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:14.643391 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb408ee-6d72-47db-a0bf-75ec50b71aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:14.644073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb408ee-6d72-47db-a0bf-75ec50b71aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:14.644401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb408ee-6d72-47db-a0bf-75ec50b71aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:14.644770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb408ee-6d72-47db-a0bf-75ec50b71aa1 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:14.645619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb408ee-6d72-47db-a0bf-75ec50b71aa1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:14.645945 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb408ee-6d72-47db-a0bf-75ec50b71aa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:14.647421 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb408ee-6d72-47db-a0bf-75ec50b71aa1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:14.657447 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 189/398] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:49:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:14.670382 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9f7ec-6b59-4f41-bf06-b4f8d84bd5fa None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:14.779045 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9f7ec-6b59-4f41-bf06-b4f8d84bd5fa None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:14.816138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef072cb1-ec57-485f-acb6-f0a70e166f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:14.817340 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef072cb1-ec57-485f-acb6-f0a70e166f70 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:14.817811 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef072cb1-ec57-485f-acb6-f0a70e166f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:14.818237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef072cb1-ec57-485f-acb6-f0a70e166f70 None admin] PATH_INFO: `/v3/groups/c93c0d034ea446718a1169bb53fc913d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:14.819659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef072cb1-ec57-485f-acb6-f0a70e166f70 None admin] RBAC: Authorizing `identity:get_group(group_id=c93c0d034ea446718a1169bb53fc913d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:14.821320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef072cb1-ec57-485f-acb6-f0a70e166f70 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:14.823490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 210/399] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 14 18:49:14 2022] GET /identity/v3/groups/c93c0d034ea446718a1169bb53fc913d => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:14.832549 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb408ee-6d72-47db-a0bf-75ec50b71aa1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:14.851050 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:14.946741 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb408ee-6d72-47db-a0bf-75ec50b71aa1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:14.988860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c99b9-794b-4f43-a6f6-1f8cf29ee3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:14.989849 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c99b9-794b-4f43-a6f6-1f8cf29ee3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:14.990142 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c99b9-794b-4f43-a6f6-1f8cf29ee3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:14.990531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c99b9-794b-4f43-a6f6-1f8cf29ee3d6 None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:14.996795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c99b9-794b-4f43-a6f6-1f8cf29ee3d6 None admin] RBAC: Authorizing `identity:get_project(project_id=45bac146c9d842bf82503d3f2b145128)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:14.996795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c99b9-794b-4f43-a6f6-1f8cf29ee3d6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:14.997347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 190/400] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:49:14 2022] GET /identity/v3/projects/45bac146c9d842bf82503d3f2b145128 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:15.006058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef072cb1-ec57-485f-acb6-f0a70e166f70 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:15.021400 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:15.084489 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1070d-1110-4e71-a231-a6b80f147332 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:15.085418 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1070d-1110-4e71-a231-a6b80f147332 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:15.086179 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1070d-1110-4e71-a231-a6b80f147332 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:15.093221 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 191/401] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:49:15.098577 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da28445-b249-49e5-8c57-a7806c19b60f None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:15.099394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da28445-b249-49e5-8c57-a7806c19b60f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:15.100178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da28445-b249-49e5-8c57-a7806c19b60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:15.112546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef072cb1-ec57-485f-acb6-f0a70e166f70 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:15.131492 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4da28445-b249-49e5-8c57-a7806c19b60f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:15.134361 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4da28445-b249-49e5-8c57-a7806c19b60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:15.152423 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a2f81-a5c0-45ad-8a92-24f60f2105d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:15.157680 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a2f81-a5c0-45ad-8a92-24f60f2105d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:15.157680 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a2f81-a5c0-45ad-8a92-24f60f2105d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:15.157680 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a2f81-a5c0-45ad-8a92-24f60f2105d6 None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128/groups/c93c0d034ea446718a1169bb53fc913d/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:15.157680 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a2f81-a5c0-45ad-8a92-24f60f2105d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=45bac146c9d842bf82503d3f2b145128, group_id=c93c0d034ea446718a1169bb53fc913d, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:15.158167 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a2f81-a5c0-45ad-8a92-24f60f2105d6 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:15.237498 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 211/402] 10.222.0.69 () {58 vars in 1442 bytes} [Thu Jul 14 18:49:15 2022] PUT /identity/v3/projects/45bac146c9d842bf82503d3f2b145128/groups/c93c0d034ea446718a1169bb53fc913d/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:49:15.270063 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 192/403] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:49:15.302756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a2f81-a5c0-45ad-8a92-24f60f2105d6 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:15.419658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:15.454952 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a2f81-a5c0-45ad-8a92-24f60f2105d6 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:15.497904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddaaa7-7d73-4e59-8ba1-b856acdff05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:15.498721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddaaa7-7d73-4e59-8ba1-b856acdff05c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:15.499016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddaaa7-7d73-4e59-8ba1-b856acdff05c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:15.499336 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddaaa7-7d73-4e59-8ba1-b856acdff05c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:15.512184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bddaaa7-7d73-4e59-8ba1-b856acdff05c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:15.520916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-4bddaaa7-7d73-4e59-8ba1-b856acdff05c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:49:15.520916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:15.520916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:15.520916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:15.520916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:15.520916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:15.520916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:15.520916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:15.520916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:15.520916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:15.520916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:15.520916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:15.520916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:15.520916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:15.520916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:15.520916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:15.521795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:15.521795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:15.521795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:15.521795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:15.521795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:15.521795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:15.521795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:15.521795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:15.521795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:15.521795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:15.521795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:15.521795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:15.521795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:15.521795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:15.521795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:15.521795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:15.523015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:15.523015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:15.523015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:15.523015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:15.523015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:15.523015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:15.523015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:15.523015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:15.523015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:15.523015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:15.523015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:49:15.523015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:15.523015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 212/404] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:49:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:15.530472 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da28445-b249-49e5-8c57-a7806c19b60f None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:15.551562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:15.658923 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da28445-b249-49e5-8c57-a7806c19b60f None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:15.697098 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab2c17-252a-4520-8007-222a46937145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:15.698173 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab2c17-252a-4520-8007-222a46937145 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:15.698647 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab2c17-252a-4520-8007-222a46937145 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:15.699108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab2c17-252a-4520-8007-222a46937145 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:15.700075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab2c17-252a-4520-8007-222a46937145 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:15.700572 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab2c17-252a-4520-8007-222a46937145 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:15.702141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab2c17-252a-4520-8007-222a46937145 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:15.715270 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 193/405] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:15.717637 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddaaa7-7d73-4e59-8ba1-b856acdff05c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:15.750065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9e875-b71c-4617-86e6-870f1363aaf4 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:15.750309 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9e875-b71c-4617-86e6-870f1363aaf4 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:15.750548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9e875-b71c-4617-86e6-870f1363aaf4 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:15.751978 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 194/406] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:49:15.757267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ac09f-1ee3-4862-8065-e1bbf674ab09 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:15.757563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ac09f-1ee3-4862-8065-e1bbf674ab09 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:15.757855 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ac09f-1ee3-4862-8065-e1bbf674ab09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:15.785904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a68ac09f-1ee3-4862-8065-e1bbf674ab09 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:15.787123 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a68ac09f-1ee3-4862-8065-e1bbf674ab09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:15.867060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddaaa7-7d73-4e59-8ba1-b856acdff05c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:15.902207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac241477-32d7-4c1d-9cf3-06bf4ab8415a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:15.903505 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac241477-32d7-4c1d-9cf3-06bf4ab8415a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:15.903985 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac241477-32d7-4c1d-9cf3-06bf4ab8415a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:15.904446 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac241477-32d7-4c1d-9cf3-06bf4ab8415a None admin] PATH_INFO: `/v3/groups/c93c0d034ea446718a1169bb53fc913d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:15.905803 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac241477-32d7-4c1d-9cf3-06bf4ab8415a None admin] RBAC: Authorizing `identity:get_group(group_id=c93c0d034ea446718a1169bb53fc913d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:15.907581 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac241477-32d7-4c1d-9cf3-06bf4ab8415a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:15.914535 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 213/407] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 14 18:49:15 2022] GET /identity/v3/groups/c93c0d034ea446718a1169bb53fc913d => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:15.915963 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac241477-32d7-4c1d-9cf3-06bf4ab8415a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:15.920819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 195/408] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:49:15.943517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:15.960231 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ac09f-1ee3-4862-8065-e1bbf674ab09 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:15.962362 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:16.068488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac241477-32d7-4c1d-9cf3-06bf4ab8415a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:16.138672 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39314359-37f2-4154-a5a8-8bca16b5ba70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:16.139788 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39314359-37f2-4154-a5a8-8bca16b5ba70 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:16.140099 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39314359-37f2-4154-a5a8-8bca16b5ba70 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:16.140495 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39314359-37f2-4154-a5a8-8bca16b5ba70 None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:16.144097 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39314359-37f2-4154-a5a8-8bca16b5ba70 None admin] RBAC: Authorizing `identity:get_project(project_id=45bac146c9d842bf82503d3f2b145128)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:16.146576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39314359-37f2-4154-a5a8-8bca16b5ba70 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:16.159251 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 214/409] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:49:15 2022] GET /identity/v3/projects/45bac146c9d842bf82503d3f2b145128 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:16.160232 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39314359-37f2-4154-a5a8-8bca16b5ba70 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:16.161537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ac09f-1ee3-4862-8065-e1bbf674ab09 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:16.200596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf7382-9cf8-406c-a47d-fdb73af43818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:16.201790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf7382-9cf8-406c-a47d-fdb73af43818 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:16.202290 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf7382-9cf8-406c-a47d-fdb73af43818 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:16.202825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf7382-9cf8-406c-a47d-fdb73af43818 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:16.215475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf7382-9cf8-406c-a47d-fdb73af43818 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:16.224511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-94cf7382-9cf8-406c-a47d-fdb73af43818 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:49:16.224511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:16.224511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:16.224511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:16.224511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:16.224511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:16.224511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:16.224511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:16.224511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:16.224511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:16.224511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:16.224511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:16.224511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:16.224511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:16.224511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:16.224511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:16.225444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:16.225444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:16.225444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:16.225444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:16.225444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:16.225444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:16.225444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:16.225444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:16.225444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:16.225444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:16.225444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:16.225444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:16.225444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:16.225444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:16.225444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:16.225444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:16.226275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:16.226275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:16.226275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:16.226275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:16.226275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:16.226275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:16.226275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:16.226275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:16.226275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:16.226275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:16.226275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:49:16.226275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:16.229240 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 196/410] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:49:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:16.235047 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf7382-9cf8-406c-a47d-fdb73af43818 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:16.308616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:16.349962 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39314359-37f2-4154-a5a8-8bca16b5ba70 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:16.357107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:16.399334 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf7382-9cf8-406c-a47d-fdb73af43818 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:16.406811 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0ec7c-185b-44b2-93c3-dd23cd583eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:16.407738 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0ec7c-185b-44b2-93c3-dd23cd583eec None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:16.408024 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0ec7c-185b-44b2-93c3-dd23cd583eec None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:16.408301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0ec7c-185b-44b2-93c3-dd23cd583eec None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128/groups/c93c0d034ea446718a1169bb53fc913d/roles/394e165201dc41c8b3255de480821b1f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:16.414284 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0ec7c-185b-44b2-93c3-dd23cd583eec None admin] RBAC: Authorizing `identity:create_grant(project_id=45bac146c9d842bf82503d3f2b145128, group_id=c93c0d034ea446718a1169bb53fc913d, role_id=394e165201dc41c8b3255de480821b1f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:16.415834 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0ec7c-185b-44b2-93c3-dd23cd583eec None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:16.449305 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea64f73-4165-43ba-83a4-21b952f018f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:16.449978 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea64f73-4165-43ba-83a4-21b952f018f3 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:16.450367 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea64f73-4165-43ba-83a4-21b952f018f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:16.450653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea64f73-4165-43ba-83a4-21b952f018f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:16.451429 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea64f73-4165-43ba-83a4-21b952f018f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:16.451744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea64f73-4165-43ba-83a4-21b952f018f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:16.453219 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea64f73-4165-43ba-83a4-21b952f018f3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:16.462456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 197/411] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:16.468136 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea64f73-4165-43ba-83a4-21b952f018f3 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:16.573082 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea64f73-4165-43ba-83a4-21b952f018f3 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:16.587931 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 215/412] 10.222.0.69 () {58 vars in 1442 bytes} [Thu Jul 14 18:49:16 2022] PUT /identity/v3/projects/45bac146c9d842bf82503d3f2b145128/groups/c93c0d034ea446718a1169bb53fc913d/roles/394e165201dc41c8b3255de480821b1f => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:49:16.590450 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad25ff5-1418-4d79-bce4-0edd5cb1051e None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:16.590961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad25ff5-1418-4d79-bce4-0edd5cb1051e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:16.591533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad25ff5-1418-4d79-bce4-0edd5cb1051e None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:16.594139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 216/413] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:16.605057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e09e8-236c-405c-a925-c7cfe44f3869 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:16.605377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e09e8-236c-405c-a925-c7cfe44f3869 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:16.605627 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e09e8-236c-405c-a925-c7cfe44f3869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:16.623061 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:16.639888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3c1e09e8-236c-405c-a925-c7cfe44f3869 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:16.641130 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3c1e09e8-236c-405c-a925-c7cfe44f3869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:16.648501 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2147907-40a2-44ad-94e6-1f82f0d20fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:16.649391 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2147907-40a2-44ad-94e6-1f82f0d20fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:16.649744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2147907-40a2-44ad-94e6-1f82f0d20fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:16.650034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2147907-40a2-44ad-94e6-1f82f0d20fd7 None admin] PATH_INFO: `/v3/groups/708197382e774537a8f99f8da9b48d99` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:16.651720 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2147907-40a2-44ad-94e6-1f82f0d20fd7 None admin] RBAC: Authorizing `identity:get_group(group_id=708197382e774537a8f99f8da9b48d99)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:16.653235 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2147907-40a2-44ad-94e6-1f82f0d20fd7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:16.659969 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 198/414] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 14 18:49:16 2022] GET /identity/v3/groups/708197382e774537a8f99f8da9b48d99 => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:16.666263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2147907-40a2-44ad-94e6-1f82f0d20fd7 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:16.801601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 217/415] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:49:16.824189 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2147907-40a2-44ad-94e6-1f82f0d20fd7 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:16.837360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e09e8-236c-405c-a925-c7cfe44f3869 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:16.879428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071aa546-d58d-4f3c-9f6d-67257b01e926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:16.880641 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071aa546-d58d-4f3c-9f6d-67257b01e926 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:16.881127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071aa546-d58d-4f3c-9f6d-67257b01e926 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:16.881641 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071aa546-d58d-4f3c-9f6d-67257b01e926 None admin] PATH_INFO: `/v3/projects/1ecc6a6cebab48d4b8cfff094ca5066b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:16.883066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071aa546-d58d-4f3c-9f6d-67257b01e926 None admin] RBAC: Authorizing `identity:get_project(project_id=1ecc6a6cebab48d4b8cfff094ca5066b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:16.885419 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071aa546-d58d-4f3c-9f6d-67257b01e926 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:16.888380 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 199/416] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:49:16 2022] GET /identity/v3/projects/1ecc6a6cebab48d4b8cfff094ca5066b => generated 344 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:16.899660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071aa546-d58d-4f3c-9f6d-67257b01e926 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:16.931489 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:16.999752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:17.013402 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e09e8-236c-405c-a925-c7cfe44f3869 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:17.063001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071aa546-d58d-4f3c-9f6d-67257b01e926 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:17.078443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf8565-75c4-4ad0-ae12-db51a1370a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:17.080109 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf8565-75c4-4ad0-ae12-db51a1370a5a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:17.080399 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf8565-75c4-4ad0-ae12-db51a1370a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:17.080676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf8565-75c4-4ad0-ae12-db51a1370a5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:17.102277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf8565-75c4-4ad0-ae12-db51a1370a5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:17.111901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-05cf8565-75c4-4ad0-ae12-db51a1370a5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:49:17.111901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:17.111901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:17.111901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:17.111901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:17.111901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:17.111901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:17.111901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:17.111901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:17.111901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:17.111901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:17.111901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:17.111901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:17.111901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:17.111901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:17.111901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:17.112797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:17.112797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:17.112797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:17.112797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:17.112797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:17.112797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:17.112797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:17.112797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:17.112797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:17.112797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:17.112797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:17.112797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:17.112797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:17.112797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:17.112797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:17.112797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:17.113608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:17.113608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:17.113608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:17.113608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:17.113608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:17.113608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:17.113608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:17.113608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:17.113608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:17.113608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:17.113608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:49:17.113608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:17.115256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 218/417] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:49:17.118812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf8565-75c4-4ad0-ae12-db51a1370a5a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:17.121922 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b1c0d-57bb-429d-b2fe-74c02515e329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:17.122833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b1c0d-57bb-429d-b2fe-74c02515e329 None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:17.123123 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b1c0d-57bb-429d-b2fe-74c02515e329 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:17.123435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b1c0d-57bb-429d-b2fe-74c02515e329 None admin] PATH_INFO: `/v3/projects/1ecc6a6cebab48d4b8cfff094ca5066b/groups/708197382e774537a8f99f8da9b48d99/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:17.125485 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b1c0d-57bb-429d-b2fe-74c02515e329 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ecc6a6cebab48d4b8cfff094ca5066b, group_id=708197382e774537a8f99f8da9b48d99, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:17.126947 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b1c0d-57bb-429d-b2fe-74c02515e329 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:17.222440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 200/418] 10.222.0.69 () {58 vars in 1442 bytes} [Thu Jul 14 18:49:16 2022] PUT /identity/v3/projects/1ecc6a6cebab48d4b8cfff094ca5066b/groups/708197382e774537a8f99f8da9b48d99/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:49:17.228860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6d338-e7a0-4577-838d-9da14383e7bc None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:17.229295 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6d338-e7a0-4577-838d-9da14383e7bc None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:17.229570 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6d338-e7a0-4577-838d-9da14383e7bc None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:17.231354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 201/419] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:17.237047 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b02beb-16d2-4d65-900b-0f09cd558c89 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:17.237192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b02beb-16d2-4d65-900b-0f09cd558c89 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:17.237473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b02beb-16d2-4d65-900b-0f09cd558c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:17.259794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b6b02beb-16d2-4d65-900b-0f09cd558c89 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:17.261950 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b6b02beb-16d2-4d65-900b-0f09cd558c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:17.273539 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf8565-75c4-4ad0-ae12-db51a1370a5a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:17.324767 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:17.352698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dacb18-db2f-496e-af0a-d15180d216ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:17.354145 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dacb18-db2f-496e-af0a-d15180d216ce None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:17.354439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dacb18-db2f-496e-af0a-d15180d216ce None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:17.354723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dacb18-db2f-496e-af0a-d15180d216ce None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:17.355781 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dacb18-db2f-496e-af0a-d15180d216ce None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:17.355989 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dacb18-db2f-496e-af0a-d15180d216ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:17.358791 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dacb18-db2f-496e-af0a-d15180d216ce None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:17.368940 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 219/420] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:49:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:17.374961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dacb18-db2f-496e-af0a-d15180d216ce None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:17.409629 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 202/421] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:17.433589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b02beb-16d2-4d65-900b-0f09cd558c89 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:17.499617 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dacb18-db2f-496e-af0a-d15180d216ce None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:17.504567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:17.537998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73377fb3-bb6e-4fef-ad6d-fa303dbf47ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:17.538899 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73377fb3-bb6e-4fef-ad6d-fa303dbf47ee None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:17.539414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73377fb3-bb6e-4fef-ad6d-fa303dbf47ee None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:17.539924 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73377fb3-bb6e-4fef-ad6d-fa303dbf47ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:17.585908 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b02beb-16d2-4d65-900b-0f09cd558c89 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:17.631766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:17.650121 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0235dc8-8edf-4fb3-a78d-12e4f6b510f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:17.658474 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0235dc8-8edf-4fb3-a78d-12e4f6b510f5 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:17.658474 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0235dc8-8edf-4fb3-a78d-12e4f6b510f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:17.658474 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0235dc8-8edf-4fb3-a78d-12e4f6b510f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:17.666446 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73377fb3-bb6e-4fef-ad6d-fa303dbf47ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:17.668084 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73377fb3-bb6e-4fef-ad6d-fa303dbf47ee None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:17.674690 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0235dc8-8edf-4fb3-a78d-12e4f6b510f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:17.685531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-a0235dc8-8edf-4fb3-a78d-12e4f6b510f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:49:17.685531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:17.685531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:17.685531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:17.685531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:17.685531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:17.685531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:17.685531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:17.685531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:17.685531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:17.685531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:17.685531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:17.685531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:17.685531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:17.685531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:17.685531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:17.686435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:17.686435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:17.686435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:17.686435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:17.686435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:17.686435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:17.686435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:17.686435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:17.686435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:17.686435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:17.686435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:17.686435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:17.686435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:17.686435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:17.686435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:17.686435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:17.687274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:17.687274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:17.687274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:17.687274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:17.687274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:17.687274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:17.687274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:17.687274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:17.687274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:17.687274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:17.687274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:49:17.687274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:17.691552 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 203/422] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:49:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:49:17.704146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0235dc8-8edf-4fb3-a78d-12e4f6b510f5 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:17.838166 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 220/423] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:17.844677 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad27b8-043f-49c6-91ec-36ced18380e9 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:17.845052 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad27b8-043f-49c6-91ec-36ced18380e9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:17.845400 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad27b8-043f-49c6-91ec-36ced18380e9 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:17.846901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 221/424] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:17.848088 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad27b8-043f-49c6-91ec-36ced18380e9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:17.887375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0235dc8-8edf-4fb3-a78d-12e4f6b510f5 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:17.901112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:17.927789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5644b56-c5b9-460f-97be-4c20b1cc365a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:17.928811 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5644b56-c5b9-460f-97be-4c20b1cc365a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:17.929392 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5644b56-c5b9-460f-97be-4c20b1cc365a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:17.929976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5644b56-c5b9-460f-97be-4c20b1cc365a None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:17.930983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5644b56-c5b9-460f-97be-4c20b1cc365a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:17.931599 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5644b56-c5b9-460f-97be-4c20b1cc365a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:17.933245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5644b56-c5b9-460f-97be-4c20b1cc365a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:17.944676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:17.948743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 204/425] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:17.950204 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84f0b4-a48c-452c-a6e3-657fabb961fb None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:17.950762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84f0b4-a48c-452c-a6e3-657fabb961fb None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:17.951145 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84f0b4-a48c-452c-a6e3-657fabb961fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:17.967469 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad27b8-043f-49c6-91ec-36ced18380e9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:17.969116 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-2f84f0b4-a48c-452c-a6e3-657fabb961fb None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:17.970508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2f84f0b4-a48c-452c-a6e3-657fabb961fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:18.007417 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffdf83-13db-4d7b-bf7c-e9a25763b5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:18.009154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffdf83-13db-4d7b-bf7c-e9a25763b5ee None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:18.009639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffdf83-13db-4d7b-bf7c-e9a25763b5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:18.010129 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffdf83-13db-4d7b-bf7c-e9a25763b5ee None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:18.011683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffdf83-13db-4d7b-bf7c-e9a25763b5ee None admin] RBAC: Authorizing `identity:get_project(project_id=b82ee754849c4779b408e856c429ab9b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:18.013356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffdf83-13db-4d7b-bf7c-e9a25763b5ee None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:18.015554 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 222/426] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:49:17 2022] GET /identity/v3/projects/b82ee754849c4779b408e856c429ab9b => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:18.016517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffdf83-13db-4d7b-bf7c-e9a25763b5ee None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:18.078563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 205/427] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:18.083499 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84f0b4-a48c-452c-a6e3-657fabb961fb None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:18.156650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffdf83-13db-4d7b-bf7c-e9a25763b5ee None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:18.197105 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84f0b4-a48c-452c-a6e3-657fabb961fb None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:18.199031 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:18.208148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a256d5-162c-4e00-9704-7e49bafcac64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:18.213148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a256d5-162c-4e00-9704-7e49bafcac64 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:18.213148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a256d5-162c-4e00-9704-7e49bafcac64 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:18.213148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a256d5-162c-4e00-9704-7e49bafcac64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:18.234732 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:18.235807 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e6dde-4186-40f5-8f9b-befad231aa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:18.237838 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e6dde-4186-40f5-8f9b-befad231aa37 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:18.238351 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e6dde-4186-40f5-8f9b-befad231aa37 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:18.238835 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e6dde-4186-40f5-8f9b-befad231aa37 None admin] PATH_INFO: `/v3/groups/c93c0d034ea446718a1169bb53fc913d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:18.240322 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e6dde-4186-40f5-8f9b-befad231aa37 None admin] RBAC: Authorizing `identity:get_group(group_id=c93c0d034ea446718a1169bb53fc913d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:18.242103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e6dde-4186-40f5-8f9b-befad231aa37 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:18.247522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 206/428] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 14 18:49:18 2022] GET /identity/v3/groups/c93c0d034ea446718a1169bb53fc913d => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:18.248614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e6dde-4186-40f5-8f9b-befad231aa37 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:18.305632 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a256d5-162c-4e00-9704-7e49bafcac64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:18.307842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a256d5-162c-4e00-9704-7e49bafcac64 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:18.363078 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e6dde-4186-40f5-8f9b-befad231aa37 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:18.408483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4df0d0-3f0c-4f2f-ae69-4c996b6840b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:18.409541 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4df0d0-3f0c-4f2f-ae69-4c996b6840b4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:18.409866 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4df0d0-3f0c-4f2f-ae69-4c996b6840b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:18.410258 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4df0d0-3f0c-4f2f-ae69-4c996b6840b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:18.426658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4df0d0-3f0c-4f2f-ae69-4c996b6840b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:18.434106 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 223/429] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:18.439398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a256d5-162c-4e00-9704-7e49bafcac64 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:18.439498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-5c4df0d0-3f0c-4f2f-ae69-4c996b6840b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:49:18.439498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:18.439498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:18.439498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:18.439498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:18.439498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:18.439498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:18.439498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:18.439498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:18.439498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:18.439498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:18.439498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:18.439498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:18.439498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:18.439498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:18.439498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:18.440342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:18.440342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:18.440342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:18.440342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:18.440342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:18.440342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:18.440342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:18.440342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:18.440342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:18.440342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:18.440342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:18.440342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:18.440342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:18.440342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:18.440342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:18.440342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:18.441806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:18.441806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:18.441806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:18.441806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:18.441806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:18.441806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:18.441806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:18.441806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:18.441806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:18.441806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:18.441806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 18:49:18.441806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:18.443242 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 207/430] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:18.445067 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4df0d0-3f0c-4f2f-ae69-4c996b6840b4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:18.494872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:18.497077 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:18.552249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a256d5-162c-4e00-9704-7e49bafcac64 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:18.589811 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89498188-966b-448f-ae9c-bbe5548d6546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:18.590322 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4df0d0-3f0c-4f2f-ae69-4c996b6840b4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:18.591036 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89498188-966b-448f-ae9c-bbe5548d6546 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:18.591842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89498188-966b-448f-ae9c-bbe5548d6546 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:18.592317 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89498188-966b-448f-ae9c-bbe5548d6546 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:18.593379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89498188-966b-448f-ae9c-bbe5548d6546 None admin] RBAC: Adding query filter params (group.id=c93c0d034ea446718a1169bb53fc913d, role.id=ffed4493824a4040977d2cd1d057e07b, scope.project.id=b82ee754849c4779b408e856c429ab9b) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:18.593888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89498188-966b-448f-ae9c-bbe5548d6546 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:18.595641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89498188-966b-448f-ae9c-bbe5548d6546 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:18.604698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 224/431] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 14 18:49:18 2022] GET /identity/v3/role_assignments?group.id=c93c0d034ea446718a1169bb53fc913d&scope.project.id=b82ee754849c4779b408e856c429ab9b&role.id=ffed4493824a4040977d2cd1d057e07b => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:18.606418 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89498188-966b-448f-ae9c-bbe5548d6546 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:18.655630 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb592f8-ca57-4f4e-b785-62ed47a7ad73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:18.656915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb592f8-ca57-4f4e-b785-62ed47a7ad73 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:18.657417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb592f8-ca57-4f4e-b785-62ed47a7ad73 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:18.657914 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb592f8-ca57-4f4e-b785-62ed47a7ad73 None admin] PATH_INFO: `/v3/projects/b82ee754849c4779b408e856c429ab9b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:18.660696 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb592f8-ca57-4f4e-b785-62ed47a7ad73 None admin] RBAC: Authorizing `identity:get_project(project_id=b82ee754849c4779b408e856c429ab9b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:18.662562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb592f8-ca57-4f4e-b785-62ed47a7ad73 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:18.666887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 208/432] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:49:18 2022] GET /identity/v3/projects/b82ee754849c4779b408e856c429ab9b => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:18.673657 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb592f8-ca57-4f4e-b785-62ed47a7ad73 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:18.780519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89498188-966b-448f-ae9c-bbe5548d6546 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:18.807498 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:18.838930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecb304-f12c-4287-99c5-748efd0d1784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:18.839694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecb304-f12c-4287-99c5-748efd0d1784 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:18.839922 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecb304-f12c-4287-99c5-748efd0d1784 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:18.840208 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecb304-f12c-4287-99c5-748efd0d1784 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:18.841353 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecb304-f12c-4287-99c5-748efd0d1784 None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:18.841674 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecb304-f12c-4287-99c5-748efd0d1784 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:18.843497 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecb304-f12c-4287-99c5-748efd0d1784 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:18.857725 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 225/433] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:49:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:18.866455 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecb304-f12c-4287-99c5-748efd0d1784 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:18.869804 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:18.873585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb592f8-ca57-4f4e-b785-62ed47a7ad73 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:18.921528 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01b546-a1dd-422d-ad94-1540699d2858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:18.922332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01b546-a1dd-422d-ad94-1540699d2858 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:18.922623 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01b546-a1dd-422d-ad94-1540699d2858 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:18.922907 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01b546-a1dd-422d-ad94-1540699d2858 None admin] PATH_INFO: `/v3/groups/c93c0d034ea446718a1169bb53fc913d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:18.924337 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01b546-a1dd-422d-ad94-1540699d2858 None admin] RBAC: Authorizing `identity:get_group(group_id=c93c0d034ea446718a1169bb53fc913d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:18.926099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01b546-a1dd-422d-ad94-1540699d2858 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:18.928027 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 209/434] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 14 18:49:18 2022] GET /identity/v3/groups/c93c0d034ea446718a1169bb53fc913d => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:18.934121 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01b546-a1dd-422d-ad94-1540699d2858 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:18.993392 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecb304-f12c-4287-99c5-748efd0d1784 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:19.045627 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661bf40f-a732-4c77-9243-f7a064503e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:19.046283 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bf40f-a732-4c77-9243-f7a064503e31 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:19.046573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bf40f-a732-4c77-9243-f7a064503e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:19.046852 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bf40f-a732-4c77-9243-f7a064503e31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:19.057818 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01b546-a1dd-422d-ad94-1540699d2858 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:19.101637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddcbe4-87ed-42c4-8cce-03bae77b3ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:19.102397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddcbe4-87ed-42c4-8cce-03bae77b3ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:19.102695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddcbe4-87ed-42c4-8cce-03bae77b3ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:19.103347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddcbe4-87ed-42c4-8cce-03bae77b3ab9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:19.103956 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddcbe4-87ed-42c4-8cce-03bae77b3ab9 None admin] RBAC: Adding query filter params (group.id=c93c0d034ea446718a1169bb53fc913d, role.id=394e165201dc41c8b3255de480821b1f, scope.project.id=b82ee754849c4779b408e856c429ab9b) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:19.104988 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddcbe4-87ed-42c4-8cce-03bae77b3ab9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:19.107435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddcbe4-87ed-42c4-8cce-03bae77b3ab9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:19.112008 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:19.117819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 210/435] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 14 18:49:18 2022] GET /identity/v3/role_assignments?group.id=c93c0d034ea446718a1169bb53fc913d&scope.project.id=b82ee754849c4779b408e856c429ab9b&role.id=394e165201dc41c8b3255de480821b1f => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:19.119179 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecd6d7-a6c3-4c79-8bf9-7113bb7302d2 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:19.119496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecd6d7-a6c3-4c79-8bf9-7113bb7302d2 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:19.119783 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecd6d7-a6c3-4c79-8bf9-7113bb7302d2 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:19.121685 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 211/436] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:19.127488 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11af99-c8b2-44f9-a380-161048897591 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:19.127782 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11af99-c8b2-44f9-a380-161048897591 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:19.130353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11af99-c8b2-44f9-a380-161048897591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:19.141567 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661bf40f-a732-4c77-9243-f7a064503e31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:19.143560 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661bf40f-a732-4c77-9243-f7a064503e31 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:19.157064 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0e11af99-c8b2-44f9-a380-161048897591 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:19.158295 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0e11af99-c8b2-44f9-a380-161048897591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:19.192415 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:19.243808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 212/437] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:19.269497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11af99-c8b2-44f9-a380-161048897591 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:19.283196 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 226/438] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:19.287667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661bf40f-a732-4c77-9243-f7a064503e31 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:19.381888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:19.402215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11af99-c8b2-44f9-a380-161048897591 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:19.412148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661bf40f-a732-4c77-9243-f7a064503e31 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:19.448091 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7cde9-f56f-4e39-9e7e-c4cf41f4d060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:19.448866 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7cde9-f56f-4e39-9e7e-c4cf41f4d060 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:19.451220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7cde9-f56f-4e39-9e7e-c4cf41f4d060 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:19.451220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7cde9-f56f-4e39-9e7e-c4cf41f4d060 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:19.454193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f69c9-b69b-4795-b143-1f588ae8e579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:19.455091 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f69c9-b69b-4795-b143-1f588ae8e579 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:19.455405 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f69c9-b69b-4795-b143-1f588ae8e579 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:19.455687 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f69c9-b69b-4795-b143-1f588ae8e579 None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:19.457470 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f69c9-b69b-4795-b143-1f588ae8e579 None admin] RBAC: Authorizing `identity:get_project(project_id=45bac146c9d842bf82503d3f2b145128)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:19.458864 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f69c9-b69b-4795-b143-1f588ae8e579 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:19.460641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 227/439] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:49:19 2022] GET /identity/v3/projects/45bac146c9d842bf82503d3f2b145128 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:19.461421 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7cde9-f56f-4e39-9e7e-c4cf41f4d060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:19.462966 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7f648-989b-4569-bded-3d80f121ab06 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:19.463292 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7f648-989b-4569-bded-3d80f121ab06 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:19.463571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7f648-989b-4569-bded-3d80f121ab06 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:19.468844 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 228/440] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:19.469623 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7f648-989b-4569-bded-3d80f121ab06 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:19.470014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-5fc7cde9-f56f-4e39-9e7e-c4cf41f4d060 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:49:19.470014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:19.470014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:19.470014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:19.470014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:19.470014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:19.470014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:19.470014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:19.470014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:19.470014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:19.470014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:19.470014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:19.470014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:19.470014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:19.470014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:19.470014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:19.470830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:19.470830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:19.470830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:19.470830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:19.470830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:19.470830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:19.470830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:19.470830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:19.470830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:19.470830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:19.470830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:19.470830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:19.470830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:19.470830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:19.470830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:19.470830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:19.471711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:19.471711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:19.471711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:19.471711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:19.471711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:19.471711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:19.471711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:19.471711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:19.471711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:19.471711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:19.471711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:49:19.471711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:19.471711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 213/441] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:49:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:49:19.476059 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5926c3-a76d-4a8e-a573-31964505d642 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:19.476326 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5926c3-a76d-4a8e-a573-31964505d642 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:19.476797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5926c3-a76d-4a8e-a573-31964505d642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:19.497482 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0c5926c3-a76d-4a8e-a573-31964505d642 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:19.498612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0c5926c3-a76d-4a8e-a573-31964505d642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:19.515668 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:19.576018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 214/442] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:19.591419 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5926c3-a76d-4a8e-a573-31964505d642 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:19.592586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7f648-989b-4569-bded-3d80f121ab06 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:19.626825 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e4ec6-93cf-448f-b8ea-8dde2bbf23d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:19.627709 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e4ec6-93cf-448f-b8ea-8dde2bbf23d5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:19.627996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e4ec6-93cf-448f-b8ea-8dde2bbf23d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:19.628272 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e4ec6-93cf-448f-b8ea-8dde2bbf23d5 None admin] PATH_INFO: `/v3/groups/c93c0d034ea446718a1169bb53fc913d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:19.629508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41e4ec6-93cf-448f-b8ea-8dde2bbf23d5 None admin] RBAC: Authorizing `identity:get_group(group_id=c93c0d034ea446718a1169bb53fc913d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:19.630958 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41e4ec6-93cf-448f-b8ea-8dde2bbf23d5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:19.637621 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 229/443] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 14 18:49:19 2022] GET /identity/v3/groups/c93c0d034ea446718a1169bb53fc913d => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:19.637621 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e4ec6-93cf-448f-b8ea-8dde2bbf23d5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:19.648189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:19.674937 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5926c3-a76d-4a8e-a573-31964505d642 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:19.708698 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb5f0a-047e-4daa-99d4-70a7839d4673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:19.709292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb5f0a-047e-4daa-99d4-70a7839d4673 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:19.709578 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb5f0a-047e-4daa-99d4-70a7839d4673 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:19.709858 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb5f0a-047e-4daa-99d4-70a7839d4673 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:19.710605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb5f0a-047e-4daa-99d4-70a7839d4673 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:19.710914 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb5f0a-047e-4daa-99d4-70a7839d4673 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:19.712350 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb5f0a-047e-4daa-99d4-70a7839d4673 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:19.720307 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 215/444] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:19.721097 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb5f0a-047e-4daa-99d4-70a7839d4673 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:19.732918 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e4ec6-93cf-448f-b8ea-8dde2bbf23d5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:19.736179 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:19.766250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55005a2-68c8-4af2-85b4-746eb55f7fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:19.767033 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55005a2-68c8-4af2-85b4-746eb55f7fce None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:19.767348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55005a2-68c8-4af2-85b4-746eb55f7fce None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:19.767623 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55005a2-68c8-4af2-85b4-746eb55f7fce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:19.778571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55005a2-68c8-4af2-85b4-746eb55f7fce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:19.786721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-e55005a2-68c8-4af2-85b4-746eb55f7fce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:49:19.786721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:19.786721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:19.786721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:19.786721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:19.786721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:19.786721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:19.786721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:19.786721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:19.786721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:19.786721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:19.786721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:19.786721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:19.786721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:19.786721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:19.786721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:19.787766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:19.787766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:19.787766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:19.787766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:19.787766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:19.787766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:19.787766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:19.787766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:19.787766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:19.787766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:19.787766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:19.787766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:19.787766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:19.787766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:19.787766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:19.787766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:19.788558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:19.788558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:19.788558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:19.788558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:19.788558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:19.788558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:19.788558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:19.788558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:19.788558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:19.788558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:19.788558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:49:19.788558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:19.788558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 230/445] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:49:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:19.789285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55005a2-68c8-4af2-85b4-746eb55f7fce None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:19.822797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb5f0a-047e-4daa-99d4-70a7839d4673 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:19.856219 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d1a52-1a54-455a-8696-83e246299bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:19.856875 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d1a52-1a54-455a-8696-83e246299bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:19.857162 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d1a52-1a54-455a-8696-83e246299bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:19.857439 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d1a52-1a54-455a-8696-83e246299bb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:19.857557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:19.858266 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d1a52-1a54-455a-8696-83e246299bb1 None admin] RBAC: Adding query filter params (group.id=c93c0d034ea446718a1169bb53fc913d, role.id=ffed4493824a4040977d2cd1d057e07b, scope.project.id=45bac146c9d842bf82503d3f2b145128) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:19.858580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d1a52-1a54-455a-8696-83e246299bb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:19.860040 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d1a52-1a54-455a-8696-83e246299bb1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:19.865976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 216/446] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 14 18:49:19 2022] GET /identity/v3/role_assignments?group.id=c93c0d034ea446718a1169bb53fc913d&scope.project.id=45bac146c9d842bf82503d3f2b145128&role.id=ffed4493824a4040977d2cd1d057e07b => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:19.872993 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d1a52-1a54-455a-8696-83e246299bb1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:19.893266 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55005a2-68c8-4af2-85b4-746eb55f7fce None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:19.933944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4f386-62b7-4cc4-b72d-00d41ceae487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:19.934630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4f386-62b7-4cc4-b72d-00d41ceae487 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:19.934897 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4f386-62b7-4cc4-b72d-00d41ceae487 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:19.935196 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4f386-62b7-4cc4-b72d-00d41ceae487 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:19.961595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:19.986733 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d1a52-1a54-455a-8696-83e246299bb1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:20.023888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4f386-62b7-4cc4-b72d-00d41ceae487 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:20.025375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4f386-62b7-4cc4-b72d-00d41ceae487 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:20.027860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08564203-d0ec-4c5a-aaef-98db679985e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:20.028538 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08564203-d0ec-4c5a-aaef-98db679985e0 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:20.028833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08564203-d0ec-4c5a-aaef-98db679985e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:20.029116 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08564203-d0ec-4c5a-aaef-98db679985e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:20.029876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08564203-d0ec-4c5a-aaef-98db679985e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:20.030190 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08564203-d0ec-4c5a-aaef-98db679985e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:20.031632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08564203-d0ec-4c5a-aaef-98db679985e0 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:20.040570 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 217/447] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:20.046795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08564203-d0ec-4c5a-aaef-98db679985e0 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:20.106999 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:20.139637 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 231/448] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:20.146188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4f386-62b7-4cc4-b72d-00d41ceae487 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:20.166728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08564203-d0ec-4c5a-aaef-98db679985e0 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:20.207797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d83a6b-e2a6-4c31-bc6c-60a845c93153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:20.208456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d83a6b-e2a6-4c31-bc6c-60a845c93153 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:20.208740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d83a6b-e2a6-4c31-bc6c-60a845c93153 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:20.209018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d83a6b-e2a6-4c31-bc6c-60a845c93153 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:20.218823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:20.260123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4f386-62b7-4cc4-b72d-00d41ceae487 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:20.278559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d83a6b-e2a6-4c31-bc6c-60a845c93153 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:20.280048 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d83a6b-e2a6-4c31-bc6c-60a845c93153 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:20.292461 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f83274-25ef-4e40-8748-477bd7be1bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:20.293312 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f83274-25ef-4e40-8748-477bd7be1bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:20.293592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f83274-25ef-4e40-8748-477bd7be1bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:20.293874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f83274-25ef-4e40-8748-477bd7be1bf9 None admin] PATH_INFO: `/v3/projects/45bac146c9d842bf82503d3f2b145128` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:20.295034 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f83274-25ef-4e40-8748-477bd7be1bf9 None admin] RBAC: Authorizing `identity:get_project(project_id=45bac146c9d842bf82503d3f2b145128)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:20.296557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f83274-25ef-4e40-8748-477bd7be1bf9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:20.298360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 232/449] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:49:20 2022] GET /identity/v3/projects/45bac146c9d842bf82503d3f2b145128 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:20.304261 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f83274-25ef-4e40-8748-477bd7be1bf9 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:20.344687 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:20.377470 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 218/450] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 18:49:20.383607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d83a6b-e2a6-4c31-bc6c-60a845c93153 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:20.402201 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f83274-25ef-4e40-8748-477bd7be1bf9 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:20.430058 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:20.436471 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4eb04a-4bf4-4f34-9dc4-2bbd153bc4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:20.437362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4eb04a-4bf4-4f34-9dc4-2bbd153bc4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:20.437650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4eb04a-4bf4-4f34-9dc4-2bbd153bc4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:20.437929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4eb04a-4bf4-4f34-9dc4-2bbd153bc4c4 None admin] PATH_INFO: `/v3/groups/c93c0d034ea446718a1169bb53fc913d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:20.439100 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4eb04a-4bf4-4f34-9dc4-2bbd153bc4c4 None admin] RBAC: Authorizing `identity:get_group(group_id=c93c0d034ea446718a1169bb53fc913d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:20.440568 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4eb04a-4bf4-4f34-9dc4-2bbd153bc4c4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:20.443114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 233/451] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 14 18:49:20 2022] GET /identity/v3/groups/c93c0d034ea446718a1169bb53fc913d => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:20.448750 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4eb04a-4bf4-4f34-9dc4-2bbd153bc4c4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:20.489479 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d83a6b-e2a6-4c31-bc6c-60a845c93153 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:20.522364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2b520-5c94-4ffe-a0ce-71c82f253dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:20.523238 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2b520-5c94-4ffe-a0ce-71c82f253dbd None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:20.523525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2b520-5c94-4ffe-a0ce-71c82f253dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:20.523804 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2b520-5c94-4ffe-a0ce-71c82f253dbd None admin] PATH_INFO: `/v3/projects/1ecc6a6cebab48d4b8cfff094ca5066b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:20.524990 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2b520-5c94-4ffe-a0ce-71c82f253dbd None admin] RBAC: Authorizing `identity:get_project(project_id=1ecc6a6cebab48d4b8cfff094ca5066b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:20.526680 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2b520-5c94-4ffe-a0ce-71c82f253dbd None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:20.528396 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 219/452] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:49:20 2022] GET /identity/v3/projects/1ecc6a6cebab48d4b8cfff094ca5066b => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:20.533879 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2b520-5c94-4ffe-a0ce-71c82f253dbd None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:20.547365 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4eb04a-4bf4-4f34-9dc4-2bbd153bc4c4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:20.557458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:20.581268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47a666-ac60-481d-9130-f24881d23efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:20.581934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47a666-ac60-481d-9130-f24881d23efc None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:20.582219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47a666-ac60-481d-9130-f24881d23efc None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:20.582496 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47a666-ac60-481d-9130-f24881d23efc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:20.583345 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47a666-ac60-481d-9130-f24881d23efc None admin] RBAC: Adding query filter params (group.id=c93c0d034ea446718a1169bb53fc913d, role.id=394e165201dc41c8b3255de480821b1f, scope.project.id=45bac146c9d842bf82503d3f2b145128) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:20.583658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47a666-ac60-481d-9130-f24881d23efc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:20.585116 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47a666-ac60-481d-9130-f24881d23efc None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:20.591557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 234/453] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 14 18:49:20 2022] GET /identity/v3/role_assignments?group.id=c93c0d034ea446718a1169bb53fc913d&scope.project.id=45bac146c9d842bf82503d3f2b145128&role.id=394e165201dc41c8b3255de480821b1f => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:20.635977 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2b520-5c94-4ffe-a0ce-71c82f253dbd None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:20.644413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:20.673078 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a26f7f-f394-4b83-bbca-7c913efccbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:20.673957 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a26f7f-f394-4b83-bbca-7c913efccbac None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:20.674247 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a26f7f-f394-4b83-bbca-7c913efccbac None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:20.674527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a26f7f-f394-4b83-bbca-7c913efccbac None admin] PATH_INFO: `/v3/groups/708197382e774537a8f99f8da9b48d99` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:20.676457 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a26f7f-f394-4b83-bbca-7c913efccbac None admin] RBAC: Authorizing `identity:get_group(group_id=708197382e774537a8f99f8da9b48d99)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:20.678490 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a26f7f-f394-4b83-bbca-7c913efccbac None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:20.680506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 220/454] 10.222.0.69 () {56 vars in 1180 bytes} [Thu Jul 14 18:49:20 2022] GET /identity/v3/groups/708197382e774537a8f99f8da9b48d99 => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:20.686323 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47a666-ac60-481d-9130-f24881d23efc None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:20.791397 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47a666-ac60-481d-9130-f24881d23efc None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:20.841980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd369f9a-befc-464e-902f-46784e1ca439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:20.842642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd369f9a-befc-464e-902f-46784e1ca439 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:20.842932 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd369f9a-befc-464e-902f-46784e1ca439 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:20.843241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd369f9a-befc-464e-902f-46784e1ca439 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:20.844081 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd369f9a-befc-464e-902f-46784e1ca439 None admin] RBAC: Adding query filter params (group.id=708197382e774537a8f99f8da9b48d99, role.id=27fc14631c494ca0b2dc7f708be016c2, scope.project.id=1ecc6a6cebab48d4b8cfff094ca5066b) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:20.844395 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd369f9a-befc-464e-902f-46784e1ca439 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:20.845846 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd369f9a-befc-464e-902f-46784e1ca439 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:20.852336 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 235/455] 10.222.0.69 () {56 vars in 1376 bytes} [Thu Jul 14 18:49:20 2022] GET /identity/v3/role_assignments?group.id=708197382e774537a8f99f8da9b48d99&scope.project.id=1ecc6a6cebab48d4b8cfff094ca5066b&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:23.476078 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcff6ad-ac77-4143-a7b3-a00202590093 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:23.476613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcff6ad-ac77-4143-a7b3-a00202590093 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:23.476613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcff6ad-ac77-4143-a7b3-a00202590093 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:23.478306 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 221/456] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:23.492779 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958721a7-93de-446e-bab5-1d9d07bd988c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:23.493713 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958721a7-93de-446e-bab5-1d9d07bd988c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:23.494191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958721a7-93de-446e-bab5-1d9d07bd988c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:23.520608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-958721a7-93de-446e-bab5-1d9d07bd988c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:23.522161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-958721a7-93de-446e-bab5-1d9d07bd988c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:23.559950 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:23.581140 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd465d-f61a-4560-9135-083cef4a9911 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:23.582379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd465d-f61a-4560-9135-083cef4a9911 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:23.582859 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd465d-f61a-4560-9135-083cef4a9911 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:23.584721 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 222/457] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:23.591508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6a840-f6ea-40ba-ac15-5dc14d07ba9c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:23.592075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6a840-f6ea-40ba-ac15-5dc14d07ba9c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:23.592528 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6a840-f6ea-40ba-ac15-5dc14d07ba9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:23.613601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 236/458] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:49:23.628323 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-94b6a840-f6ea-40ba-ac15-5dc14d07ba9c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:23.630074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-94b6a840-f6ea-40ba-ac15-5dc14d07ba9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:23.637830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958721a7-93de-446e-bab5-1d9d07bd988c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:23.729885 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 223/459] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:23.731496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64250f0-8fa3-458e-b560-02bcebc1af3b None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:23.731949 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64250f0-8fa3-458e-b560-02bcebc1af3b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:23.732368 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64250f0-8fa3-458e-b560-02bcebc1af3b None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:23.733778 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 224/460] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:23.739288 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223781ad-fc0a-402e-a02e-41bf82bf8f04 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:23.739763 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223781ad-fc0a-402e-a02e-41bf82bf8f04 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:23.740192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223781ad-fc0a-402e-a02e-41bf82bf8f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:23.758881 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-223781ad-fc0a-402e-a02e-41bf82bf8f04 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:23.760320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-223781ad-fc0a-402e-a02e-41bf82bf8f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:23.782845 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958721a7-93de-446e-bab5-1d9d07bd988c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:23.838138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5eb6c-e7f1-4572-8198-2c270c288931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:23.838945 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5eb6c-e7f1-4572-8198-2c270c288931 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:23.839288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5eb6c-e7f1-4572-8198-2c270c288931 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:23.839635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5eb6c-e7f1-4572-8198-2c270c288931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:23.841285 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 225/461] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:49:23.848165 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223781ad-fc0a-402e-a02e-41bf82bf8f04 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:23.853770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5eb6c-e7f1-4572-8198-2c270c288931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:23.856194 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5eb6c-e7f1-4572-8198-2c270c288931 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:23.880281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-ced5eb6c-e7f1-4572-8198-2c270c288931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:23.880281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:23.880281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:23.880281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:23.880281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:23.880281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:23.880281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:23.880281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:23.880281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:23.880281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:23.880281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:23.880281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:23.880281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:23.880281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:23.880281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:23.881375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:23.881375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:23.881375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:23.881375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:23.881375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:23.881375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:23.881375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:23.881375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:23.881375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:23.881375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:23.881375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:23.881375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:23.881375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:23.881375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:23.881375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:23.881375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:23.881375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:23.882447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:23.882447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:23.882447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:23.882447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:23.884986 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 237/462] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:49:23.886066 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5eb6c-e7f1-4572-8198-2c270c288931 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:23.903426 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:23.916861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:23.987096 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223781ad-fc0a-402e-a02e-41bf82bf8f04 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:24.015271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5eb6c-e7f1-4572-8198-2c270c288931 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:24.038523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acb82e-09db-414a-a3bf-332ccb485594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:24.039698 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acb82e-09db-414a-a3bf-332ccb485594 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:24.040190 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acb82e-09db-414a-a3bf-332ccb485594 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:24.040666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acb82e-09db-414a-a3bf-332ccb485594 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:24.052231 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acb82e-09db-414a-a3bf-332ccb485594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:24.053564 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf422e33-5e3d-4c74-8341-e1f2accf4418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:24.054364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf422e33-5e3d-4c74-8341-e1f2accf4418 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:24.054471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acb82e-09db-414a-a3bf-332ccb485594 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:24.054647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf422e33-5e3d-4c74-8341-e1f2accf4418 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:24.054924 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf422e33-5e3d-4c74-8341-e1f2accf4418 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:24.066913 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf422e33-5e3d-4c74-8341-e1f2accf4418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:24.068421 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf422e33-5e3d-4c74-8341-e1f2accf4418 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:24.074808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-d3acb82e-09db-414a-a3bf-332ccb485594 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:24.074808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:24.074808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:24.074808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:24.074808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:24.074808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:24.074808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:24.074808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:24.074808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:24.074808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:24.074808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:24.074808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:24.074808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:24.074808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:24.074808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:24.075785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:24.075785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:24.075785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:24.075785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:24.075785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:24.075785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:24.075785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:24.075785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:24.075785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:24.075785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:24.075785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:24.075785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:24.075785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:24.075785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:24.075785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:24.075785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:24.075785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:24.076793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:24.076793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:24.076793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:24.076793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:24.078086 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 226/463] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:24.078893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acb82e-09db-414a-a3bf-332ccb485594 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:24.089912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-bf422e33-5e3d-4c74-8341-e1f2accf4418 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:24.089912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:24.089912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:24.089912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:24.089912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:24.089912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:24.089912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:24.089912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:24.089912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:24.089912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:24.089912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:24.089912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:24.089912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:24.089912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:24.089912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:24.090796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:24.090796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:24.090796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:24.090796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:24.090796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:24.090796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:24.090796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:24.090796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:24.090796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:24.090796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:24.090796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:24.090796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:24.090796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:24.090796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:24.090796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:24.090796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:24.090796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:24.092065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:24.092065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:24.092065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:24.092065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:24.092065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 238/464] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:24.092545 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65a782-db8c-4621-bb21-d0b63b974d01 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:24.092718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65a782-db8c-4621-bb21-d0b63b974d01 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:24.093385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65a782-db8c-4621-bb21-d0b63b974d01 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:24.097442 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 239/465] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:24.099051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65a782-db8c-4621-bb21-d0b63b974d01 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:24.175064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:24.184179 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:24.199698 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acb82e-09db-414a-a3bf-332ccb485594 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:24.209732 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65a782-db8c-4621-bb21-d0b63b974d01 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:24.236906 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e34a2d-2e46-4843-bb4e-12159680bbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:24.238149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e34a2d-2e46-4843-bb4e-12159680bbda None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:24.238582 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e34a2d-2e46-4843-bb4e-12159680bbda None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:24.238867 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e34a2d-2e46-4843-bb4e-12159680bbda None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:24.240216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e34a2d-2e46-4843-bb4e-12159680bbda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:24.240216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e34a2d-2e46-4843-bb4e-12159680bbda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:24.241332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e34a2d-2e46-4843-bb4e-12159680bbda None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:24.248695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f9513-9883-4407-a93a-25b174976e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:24.248987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f9513-9883-4407-a93a-25b174976e3c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:24.249258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f9513-9883-4407-a93a-25b174976e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:24.249536 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f9513-9883-4407-a93a-25b174976e3c None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:24.250258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f9513-9883-4407-a93a-25b174976e3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:24.250571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f9513-9883-4407-a93a-25b174976e3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:24.252085 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f9513-9883-4407-a93a-25b174976e3c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:24.258519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 227/466] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:24.258519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e34a2d-2e46-4843-bb4e-12159680bbda None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:24.272802 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 240/467] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:24.275137 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd71153-b0d1-4c60-9a0d-afcedaec269d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:24.275942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd71153-b0d1-4c60-9a0d-afcedaec269d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:24.276693 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd71153-b0d1-4c60-9a0d-afcedaec269d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:24.297411 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-cdd71153-b0d1-4c60-9a0d-afcedaec269d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:24.298430 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cdd71153-b0d1-4c60-9a0d-afcedaec269d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:24.368119 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e34a2d-2e46-4843-bb4e-12159680bbda None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:24.369636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 241/468] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:24.370674 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd71153-b0d1-4c60-9a0d-afcedaec269d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:24.409805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd9834-6160-4c45-9271-d4b6543454c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:24.410465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd9834-6160-4c45-9271-d4b6543454c7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:24.410748 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd9834-6160-4c45-9271-d4b6543454c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:24.411023 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd9834-6160-4c45-9271-d4b6543454c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:24.411766 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd9834-6160-4c45-9271-d4b6543454c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:24.412072 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd9834-6160-4c45-9271-d4b6543454c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:24.413444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd9834-6160-4c45-9271-d4b6543454c7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:24.425957 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 228/469] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:24.427001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd9834-6160-4c45-9271-d4b6543454c7 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:24.436689 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:24.449513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:24.475132 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd71153-b0d1-4c60-9a0d-afcedaec269d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:24.509979 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39ff0a-0043-4f92-8d53-90c28f703bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:24.510576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39ff0a-0043-4f92-8d53-90c28f703bfa None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:24.510847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39ff0a-0043-4f92-8d53-90c28f703bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:24.511122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39ff0a-0043-4f92-8d53-90c28f703bfa None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:24.511851 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39ff0a-0043-4f92-8d53-90c28f703bfa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:24.513374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39ff0a-0043-4f92-8d53-90c28f703bfa None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:24.546015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd9834-6160-4c45-9271-d4b6543454c7 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:24.579728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff47891-1faf-4a52-8a25-a4e8ff5f0316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:24.580452 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff47891-1faf-4a52-8a25-a4e8ff5f0316 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:24.580681 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff47891-1faf-4a52-8a25-a4e8ff5f0316 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:24.580958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff47891-1faf-4a52-8a25-a4e8ff5f0316 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:24.582354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff47891-1faf-4a52-8a25-a4e8ff5f0316 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:24.583839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff47891-1faf-4a52-8a25-a4e8ff5f0316 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:24.609724 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 242/470] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 14 18:49:24 2022] POST /identity/v3/users => generated 246 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:49:24.610563 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39ff0a-0043-4f92-8d53-90c28f703bfa None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:24.667573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 229/471] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 14 18:49:24 2022] POST /identity/v3/users => generated 248 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:49:24.669095 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff47891-1faf-4a52-8a25-a4e8ff5f0316 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:24.740227 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39ff0a-0043-4f92-8d53-90c28f703bfa None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:24.761097 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:24.761833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:24.782716 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933d89c-d27c-42d2-aede-b5c2ffcd2593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:24.783568 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933d89c-d27c-42d2-aede-b5c2ffcd2593 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:24.783859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933d89c-d27c-42d2-aede-b5c2ffcd2593 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:24.784398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933d89c-d27c-42d2-aede-b5c2ffcd2593 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:24.800955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c933d89c-d27c-42d2-aede-b5c2ffcd2593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:24.802612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c933d89c-d27c-42d2-aede-b5c2ffcd2593 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:24.815953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff47891-1faf-4a52-8a25-a4e8ff5f0316 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:24.828857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c933d89c-d27c-42d2-aede-b5c2ffcd2593 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:24.828857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:24.828857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:24.828857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:24.828857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:24.828857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:24.828857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:24.828857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:24.828857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:24.828857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:24.828857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:24.828857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:24.828857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:24.828857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:24.828857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:24.836923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:24.836923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:24.836923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:24.836923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:24.836923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:24.836923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:24.836923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:24.836923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:24.836923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:24.836923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:24.836923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:24.836923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:24.836923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:24.836923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:24.836923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:24.836923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:24.836923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:24.837747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:24.837747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:24.837747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:24.837747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:24.837747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 243/472] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:24.837747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01151f52-4cc5-4b0a-b90a-0c225edd9e7d None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:24.837747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01151f52-4cc5-4b0a-b90a-0c225edd9e7d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:24.837747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01151f52-4cc5-4b0a-b90a-0c225edd9e7d None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:24.837747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 244/473] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:24.839159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01151f52-4cc5-4b0a-b90a-0c225edd9e7d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:24.864709 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36461ec-f702-4bc5-a80c-b568e4b2ea16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:24.865556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36461ec-f702-4bc5-a80c-b568e4b2ea16 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:24.865853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36461ec-f702-4bc5-a80c-b568e4b2ea16 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:24.866141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36461ec-f702-4bc5-a80c-b568e4b2ea16 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:24.867155 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36461ec-f702-4bc5-a80c-b568e4b2ea16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:24.868652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36461ec-f702-4bc5-a80c-b568e4b2ea16 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:24.956480 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01151f52-4cc5-4b0a-b90a-0c225edd9e7d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:24.972104 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 230/474] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 14 18:49:24 2022] POST /identity/v3/users => generated 248 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:49:24.974035 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424a857-6f82-46a3-94c6-124c11e4658b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:24.974540 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424a857-6f82-46a3-94c6-124c11e4658b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:24.975341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424a857-6f82-46a3-94c6-124c11e4658b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:24.996587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3424a857-6f82-46a3-94c6-124c11e4658b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:24.998343 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3424a857-6f82-46a3-94c6-124c11e4658b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:25.006161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee264b-60c0-47b1-8e6a-3d2a8b7273c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:25.006942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee264b-60c0-47b1-8e6a-3d2a8b7273c7 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:25.007255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee264b-60c0-47b1-8e6a-3d2a8b7273c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:25.007533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee264b-60c0-47b1-8e6a-3d2a8b7273c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:25.008328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee264b-60c0-47b1-8e6a-3d2a8b7273c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:25.008635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee264b-60c0-47b1-8e6a-3d2a8b7273c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:25.016985 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee264b-60c0-47b1-8e6a-3d2a8b7273c7 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:25.042009 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 245/475] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:25.048210 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee264b-60c0-47b1-8e6a-3d2a8b7273c7 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:25.088872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 231/476] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:25.102260 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:25.131421 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424a857-6f82-46a3-94c6-124c11e4658b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:25.203942 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:25.209628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee264b-60c0-47b1-8e6a-3d2a8b7273c7 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:25.258672 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c672a-93da-4ca4-9668-79342df9e822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:25.259381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c672a-93da-4ca4-9668-79342df9e822 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:25.259783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c672a-93da-4ca4-9668-79342df9e822 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:25.260067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c672a-93da-4ca4-9668-79342df9e822 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:25.260785 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c672a-93da-4ca4-9668-79342df9e822 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:25.262735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c672a-93da-4ca4-9668-79342df9e822 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:25.269718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424a857-6f82-46a3-94c6-124c11e4658b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:25.313399 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d79323-5aa4-4298-bdf8-1b29f143563c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:25.314076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d79323-5aa4-4298-bdf8-1b29f143563c None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:25.314367 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d79323-5aa4-4298-bdf8-1b29f143563c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:25.314646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d79323-5aa4-4298-bdf8-1b29f143563c None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:25.315443 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d79323-5aa4-4298-bdf8-1b29f143563c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:25.316864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d79323-5aa4-4298-bdf8-1b29f143563c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:25.335869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 246/477] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 14 18:49:25 2022] POST /identity/v3/users => generated 249 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:49:25.352770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.sql.core [None req-47d79323-5aa4-4298-bdf8-1b29f143563c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 18:49:25.352770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 18:49:25.352770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [parameters: {'id': 'b482f6b666024a0d92442e28ed25959d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 18:49:25.352770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80342) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 18:49:25.371734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-47d79323-5aa4-4298-bdf8-1b29f143563c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 18:49:25.371734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:25.371734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 18:49:25.371734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 18:49:25.371734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 18:49:25.371734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 18:49:25.371734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 18:49:25.371734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 18:49:25.371734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 18:49:25.371734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application conn.query(q) Jul 14 18:49:25.371734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 18:49:25.371734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 18:49:25.371734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 18:49:25.371734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application result.read() Jul 14 18:49:25.371734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 18:49:25.372648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 18:49:25.372648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 18:49:25.372648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 18:49:25.372648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 18:49:25.372648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 18:49:25.372648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 18:49:25.372648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 18:49:25.372648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 18:49:25.372648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:25.372648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 18:49:25.372648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:25.372648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:25.372648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 18:49:25.372648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 18:49:25.372648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 14 18:49:25.372648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return ref.to_dict() Jul 14 18:49:25.372648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 18:49:25.372648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application next(self.gen) Jul 14 18:49:25.373530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 18:49:25.373530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application yield resource Jul 14 18:49:25.373530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 18:49:25.373530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application next(self.gen) Jul 14 18:49:25.373530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 18:49:25.373530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 18:49:25.373530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 18:49:25.373530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 18:49:25.373530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 18:49:25.373530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise self.value Jul 14 18:49:25.373530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 18:49:25.373530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 18:49:25.373530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 18:49:25.373530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application session.commit() Jul 14 18:49:25.373530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 18:49:25.373530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 18:49:25.373530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 18:49:25.374378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 18:49:25.374378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 18:49:25.374378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self.session.flush() Jul 14 18:49:25.374378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 18:49:25.374378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 18:49:25.374378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 18:49:25.374378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 18:49:25.374378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 18:49:25.374378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application compat.raise_( Jul 14 18:49:25.374378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 18:49:25.374378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception Jul 14 18:49:25.374378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 18:49:25.374378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 18:49:25.374378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 18:49:25.374378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 18:49:25.374378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 18:49:25.374378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 18:49:25.375294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 18:49:25.375294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 18:49:25.375294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 18:49:25.375294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 18:49:25.375294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 18:49:25.375294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 18:49:25.375294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 18:49:25.375294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 18:49:25.375294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 18:49:25.375294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 18:49:25.375294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 18:49:25.375294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 18:49:25.375294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 18:49:25.375294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 18:49:25.375294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 18:49:25.375294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception Jul 14 18:49:25.377160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 18:49:25.377160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 18:49:25.377160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 18:49:25.377160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 18:49:25.377160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 18:49:25.377160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 18:49:25.377160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 18:49:25.377160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application conn.query(q) Jul 14 18:49:25.377160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 18:49:25.377160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 18:49:25.377160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 18:49:25.377160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application result.read() Jul 14 18:49:25.377160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 18:49:25.377160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 18:49:25.377160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 18:49:25.377160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 18:49:25.379076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 18:49:25.379076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 18:49:25.379076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 18:49:25.379076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 18:49:25.379076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 18:49:25.379076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 18:49:25.379076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [parameters: {'id': 'b482f6b666024a0d92442e28ed25959d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 18:49:25.379076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 18:49:25.379076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:25.379076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:49:25.379076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:25.379076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:25.379076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:25.379076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:25.379076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:25.379076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:25.380864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:25.380864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:25.380864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:25.380864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:25.380864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:25.380864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:25.380864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 14 18:49:25.380864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 14 18:49:25.380864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:25.380864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:25.380864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 14 18:49:25.380864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 14 18:49:25.380864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 18:49:25.380864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 18:49:25.380864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 18:49:25.380864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:25.382199 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 232/478] 10.222.0.69 () {60 vars in 1131 bytes} [Thu Jul 14 18:49:25 2022] POST /identity/v3/roles => generated 144 bytes in 243 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:49:25.383906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c672a-93da-4ca4-9668-79342df9e822 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:25.503872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:25.521641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c672a-93da-4ca4-9668-79342df9e822 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:25.559487 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cec261-29cd-4117-b55e-2a6fe1d4c9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:25.560554 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cec261-29cd-4117-b55e-2a6fe1d4c9fa None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:25.561003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cec261-29cd-4117-b55e-2a6fe1d4c9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:25.561486 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cec261-29cd-4117-b55e-2a6fe1d4c9fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:25.572977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cec261-29cd-4117-b55e-2a6fe1d4c9fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:25.581339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-55cec261-29cd-4117-b55e-2a6fe1d4c9fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:49:25.581339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:25.581339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:25.581339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:25.581339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:25.581339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:25.581339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:25.581339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:25.581339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:25.581339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:25.581339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:25.581339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:25.581339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:25.581339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:25.581339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:25.581339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:25.582904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:25.582904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:25.582904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:25.582904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:25.582904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:25.582904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:25.582904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:25.582904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:25.582904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:25.582904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:25.582904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:25.582904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:25.582904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:25.582904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:25.582904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:25.582904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:25.583892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:25.583892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:25.583892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:25.583892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:25.583892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:25.583892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:25.583892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:25.583892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:25.583892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:25.583892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:25.583892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 18:49:25.583892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:25.586247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 247/479] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:49:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:25.592116 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d79323-5aa4-4298-bdf8-1b29f143563c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:25.699074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d79323-5aa4-4298-bdf8-1b29f143563c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:25.712531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:25.734448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa14862-13cb-4f80-83d4-0a8d527e3ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:25.735129 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa14862-13cb-4f80-83d4-0a8d527e3ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:25.735467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa14862-13cb-4f80-83d4-0a8d527e3ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:25.735749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa14862-13cb-4f80-83d4-0a8d527e3ae4 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:25.736502 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa14862-13cb-4f80-83d4-0a8d527e3ae4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:25.736814 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa14862-13cb-4f80-83d4-0a8d527e3ae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:25.738318 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa14862-13cb-4f80-83d4-0a8d527e3ae4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:25.747461 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 233/480] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:27.050781 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a862b-201c-4f5c-86ff-88d77041eff2 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:27.051747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a862b-201c-4f5c-86ff-88d77041eff2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:27.052250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a862b-201c-4f5c-86ff-88d77041eff2 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:27.053904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 248/481] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:27.059246 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57858b79-99f5-4ffb-9772-bc4de925fcd1 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:27.059657 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57858b79-99f5-4ffb-9772-bc4de925fcd1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:27.059826 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57858b79-99f5-4ffb-9772-bc4de925fcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:27.081024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-57858b79-99f5-4ffb-9772-bc4de925fcd1 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:27.082762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-57858b79-99f5-4ffb-9772-bc4de925fcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:27.106773 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152832e2-d169-43ff-a5c3-f2665117742c None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:27.107067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152832e2-d169-43ff-a5c3-f2665117742c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:27.107400 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152832e2-d169-43ff-a5c3-f2665117742c None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:27.109040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 249/482] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:27.114128 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06d05f-d34e-41bb-ace8-3f0b1ad6d857 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:27.114416 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06d05f-d34e-41bb-ace8-3f0b1ad6d857 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:27.114697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06d05f-d34e-41bb-ace8-3f0b1ad6d857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:27.131058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2c06d05f-d34e-41bb-ace8-3f0b1ad6d857 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:27.132282 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2c06d05f-d34e-41bb-ace8-3f0b1ad6d857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:27.180260 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 234/483] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:27.211339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 250/484] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:27.219083 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57858b79-99f5-4ffb-9772-bc4de925fcd1 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:27.248481 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06d05f-d34e-41bb-ace8-3f0b1ad6d857 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:27.370449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57858b79-99f5-4ffb-9772-bc4de925fcd1 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:27.375479 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:27.396846 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:27.430090 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb4bd8a-d453-42b4-9746-b6fc810468c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:27.430948 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4bd8a-d453-42b4-9746-b6fc810468c8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:27.431281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4bd8a-d453-42b4-9746-b6fc810468c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:27.431573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4bd8a-d453-42b4-9746-b6fc810468c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:27.443373 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb4bd8a-d453-42b4-9746-b6fc810468c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:27.451567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-7eb4bd8a-d453-42b4-9746-b6fc810468c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:27.451567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:27.451567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:27.451567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:27.451567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:27.451567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:27.451567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:27.451567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:27.451567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:27.451567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:27.451567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:27.451567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:27.451567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:27.451567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:27.451567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:27.451567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:27.452435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:27.452435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:27.452435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:27.452435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:27.452435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:27.452435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:27.452435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:27.452435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:27.452435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:27.452435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:27.452435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:27.452435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:27.452435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:27.452435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:27.452435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:27.452435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:27.453244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:27.453244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:27.453244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:27.453244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:27.453244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:27.453244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:27.453244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:27.453244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:27.453244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:27.453244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:27.453244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:27.453244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:27.453244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 235/485] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:27.459959 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb4bd8a-d453-42b4-9746-b6fc810468c8 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:27.464322 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06d05f-d34e-41bb-ace8-3f0b1ad6d857 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:27.530541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30353d34-e274-4499-99db-eb206eb09068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:27.531420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30353d34-e274-4499-99db-eb206eb09068 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:27.531712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30353d34-e274-4499-99db-eb206eb09068 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:27.531991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30353d34-e274-4499-99db-eb206eb09068 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:27.542850 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30353d34-e274-4499-99db-eb206eb09068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:27.552663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-30353d34-e274-4499-99db-eb206eb09068 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:27.552663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:27.552663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:27.552663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:27.552663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:27.552663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:27.552663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:27.552663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:27.552663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:27.552663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:27.552663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:27.552663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:27.552663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:27.552663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:27.552663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:27.552663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:27.553541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:27.553541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:27.553541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:27.553541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:27.553541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:27.553541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:27.553541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:27.553541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:27.553541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:27.553541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:27.553541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:27.553541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:27.553541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:27.553541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:27.553541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:27.553541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:27.554348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:27.554348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:27.554348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:27.554348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:27.554348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:27.554348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:27.554348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:27.554348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:27.554348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:27.554348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:27.554348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:27.554348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:27.554348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 251/486] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:49:27.555245 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5098e0-10d7-4bfe-b573-4f2b549c0b18 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:27.555493 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5098e0-10d7-4bfe-b573-4f2b549c0b18 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:27.555796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5098e0-10d7-4bfe-b573-4f2b549c0b18 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:27.557104 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 252/487] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:27.561736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5098e0-10d7-4bfe-b573-4f2b549c0b18 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:27.568850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb4bd8a-d453-42b4-9746-b6fc810468c8 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:27.604046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ad59b-0d98-4a73-b12b-7edad9fe3bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:27.604808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ad59b-0d98-4a73-b12b-7edad9fe3bbd None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:27.605167 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ad59b-0d98-4a73-b12b-7edad9fe3bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:27.605517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ad59b-0d98-4a73-b12b-7edad9fe3bbd None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:27.606351 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ad59b-0d98-4a73-b12b-7edad9fe3bbd None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:27.606737 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ad59b-0d98-4a73-b12b-7edad9fe3bbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:27.608213 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ad59b-0d98-4a73-b12b-7edad9fe3bbd None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:27.616799 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 236/488] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:49:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:27.618085 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b6416-7240-4c9c-abb5-0f52d84120fe None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:27.623138 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b6416-7240-4c9c-abb5-0f52d84120fe None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:27.623485 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b6416-7240-4c9c-abb5-0f52d84120fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:27.640305 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c42b6416-7240-4c9c-abb5-0f52d84120fe None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:27.641573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c42b6416-7240-4c9c-abb5-0f52d84120fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:27.663058 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:27.673439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5098e0-10d7-4bfe-b573-4f2b549c0b18 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:27.709672 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe078e-c449-4ae5-b2b9-1177e2e3590c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:27.710834 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe078e-c449-4ae5-b2b9-1177e2e3590c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:27.711329 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe078e-c449-4ae5-b2b9-1177e2e3590c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:27.719186 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 237/489] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:27.719186 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b6416-7240-4c9c-abb5-0f52d84120fe None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:27.719773 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe078e-c449-4ae5-b2b9-1177e2e3590c None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:27.721291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe078e-c449-4ae5-b2b9-1177e2e3590c None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:27.721782 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe078e-c449-4ae5-b2b9-1177e2e3590c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:27.723522 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe078e-c449-4ae5-b2b9-1177e2e3590c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:27.725308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:27.743086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 253/490] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:49:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:27.751924 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe078e-c449-4ae5-b2b9-1177e2e3590c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:27.816617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b6416-7240-4c9c-abb5-0f52d84120fe None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:27.851465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf21912-38f8-4895-865d-bfcee8e69837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:27.852261 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf21912-38f8-4895-865d-bfcee8e69837 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:27.852778 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf21912-38f8-4895-865d-bfcee8e69837 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:27.852893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf21912-38f8-4895-865d-bfcee8e69837 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:27.862065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe078e-c449-4ae5-b2b9-1177e2e3590c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:27.863846 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf21912-38f8-4895-865d-bfcee8e69837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:27.866008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf21912-38f8-4895-865d-bfcee8e69837 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:27.884125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-6cf21912-38f8-4895-865d-bfcee8e69837 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:27.884125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:27.884125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:27.884125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:27.884125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:27.884125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:27.884125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:27.884125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:27.884125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:27.884125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:27.884125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:27.884125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:27.884125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:27.884125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:27.884125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:27.885154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:27.885154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:27.885154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:27.885154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:27.885154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:27.885154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:27.885154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:27.885154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:27.885154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:27.885154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:27.885154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:27.885154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:27.885154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:27.885154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:27.885154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:27.885154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:27.885154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:27.886201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:27.886201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:27.886201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:27.886201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:27.886201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 238/491] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:27.886533 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf21912-38f8-4895-865d-bfcee8e69837 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:27.902164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:27.904833 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf89c2-5d66-4aa7-8f20-f9c1ef3aeb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:27.905802 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf89c2-5d66-4aa7-8f20-f9c1ef3aeb26 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:27.906261 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf89c2-5d66-4aa7-8f20-f9c1ef3aeb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:27.906678 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf89c2-5d66-4aa7-8f20-f9c1ef3aeb26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:27.918025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf89c2-5d66-4aa7-8f20-f9c1ef3aeb26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:27.919822 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf89c2-5d66-4aa7-8f20-f9c1ef3aeb26 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:27.940138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-1edf89c2-5d66-4aa7-8f20-f9c1ef3aeb26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:27.940138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:27.940138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:27.940138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:27.940138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:27.940138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:27.940138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:27.940138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:27.940138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:27.940138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:27.940138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:27.940138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:27.940138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:27.940138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:27.940138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:27.941792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:27.941792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:27.941792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:27.941792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:27.941792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:27.941792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:27.941792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:27.941792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:27.941792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:27.941792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:27.941792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:27.941792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:27.941792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:27.941792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:27.941792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:27.941792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:27.941792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:27.942667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:27.942667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:27.942667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:27.942667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:27.944600 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 254/492] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:49:27.945524 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf89c2-5d66-4aa7-8f20-f9c1ef3aeb26 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:27.997774 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf21912-38f8-4895-865d-bfcee8e69837 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:27.999004 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:28.039960 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd51b7e-1cb1-44aa-ac26-b87d3c59cb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:28.041016 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd51b7e-1cb1-44aa-ac26-b87d3c59cb1a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:28.041534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd51b7e-1cb1-44aa-ac26-b87d3c59cb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:28.041956 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd51b7e-1cb1-44aa-ac26-b87d3c59cb1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:28.053339 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd51b7e-1cb1-44aa-ac26-b87d3c59cb1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:28.056031 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf89c2-5d66-4aa7-8f20-f9c1ef3aeb26 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:28.061743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-4cd51b7e-1cb1-44aa-ac26-b87d3c59cb1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:28.061743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:28.061743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:28.061743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:28.061743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:28.061743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:28.061743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:28.061743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:28.061743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:28.061743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:28.061743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:28.061743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:28.061743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:28.061743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:28.061743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:28.061743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:28.062984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:28.062984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:28.062984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:28.062984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:28.062984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:28.062984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:28.062984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:28.062984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:28.062984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:28.062984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:28.062984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:28.062984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:28.062984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:28.062984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:28.062984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:28.062984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:28.063790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:28.063790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:28.063790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:28.063790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:28.063790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:28.063790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:28.063790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:28.063790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:28.063790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:28.063790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:28.063790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:28.063790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:28.066184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 239/493] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:28.067192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd51b7e-1cb1-44aa-ac26-b87d3c59cb1a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:28.098264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d834a-612c-420f-b018-f25b5f4b4e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:28.098836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d834a-612c-420f-b018-f25b5f4b4e3e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:28.099123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d834a-612c-420f-b018-f25b5f4b4e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:28.099447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d834a-612c-420f-b018-f25b5f4b4e3e None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:28.100175 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d834a-612c-420f-b018-f25b5f4b4e3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:28.100487 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d834a-612c-420f-b018-f25b5f4b4e3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:28.102098 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d834a-612c-420f-b018-f25b5f4b4e3e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:28.114998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 255/494] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:28.115868 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d834a-612c-420f-b018-f25b5f4b4e3e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:28.139047 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:28.174110 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd51b7e-1cb1-44aa-ac26-b87d3c59cb1a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:28.216563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a6238-d575-4b69-8e2a-501ed7c840a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:28.217236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a6238-d575-4b69-8e2a-501ed7c840a2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:28.217723 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a6238-d575-4b69-8e2a-501ed7c840a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:28.217846 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a6238-d575-4b69-8e2a-501ed7c840a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:28.218510 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a6238-d575-4b69-8e2a-501ed7c840a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:28.218820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a6238-d575-4b69-8e2a-501ed7c840a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:28.220257 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a6238-d575-4b69-8e2a-501ed7c840a2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:28.224600 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d834a-612c-420f-b018-f25b5f4b4e3e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:28.234186 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:28.234832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 240/495] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:28.236595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a6238-d575-4b69-8e2a-501ed7c840a2 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:28.259818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84655549-6fc6-4af8-bd62-1bda86ca5cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:28.260491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84655549-6fc6-4af8-bd62-1bda86ca5cda None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:28.260779 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84655549-6fc6-4af8-bd62-1bda86ca5cda None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:28.261060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84655549-6fc6-4af8-bd62-1bda86ca5cda None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:28.261807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84655549-6fc6-4af8-bd62-1bda86ca5cda None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:28.262119 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84655549-6fc6-4af8-bd62-1bda86ca5cda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:28.263543 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84655549-6fc6-4af8-bd62-1bda86ca5cda None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:28.272284 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 256/496] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:49:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:28.273456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46a410-e702-4ab0-ae26-84911fb7fccc None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:28.273751 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46a410-e702-4ab0-ae26-84911fb7fccc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:28.274032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46a410-e702-4ab0-ae26-84911fb7fccc None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:28.279692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 257/497] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:28.279692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46a410-e702-4ab0-ae26-84911fb7fccc None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:28.355453 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a6238-d575-4b69-8e2a-501ed7c840a2 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:28.380071 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46a410-e702-4ab0-ae26-84911fb7fccc None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:28.381299 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:28.388564 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6414b-b310-4d10-905f-c5d3353546e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:28.396094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6414b-b310-4d10-905f-c5d3353546e5 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:28.396094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6414b-b310-4d10-905f-c5d3353546e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:28.396094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6414b-b310-4d10-905f-c5d3353546e5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:28.397393 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6414b-b310-4d10-905f-c5d3353546e5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:28.398872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6414b-b310-4d10-905f-c5d3353546e5 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:28.413495 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1cfdb-32c6-4d92-8c1d-588860eb5686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:28.414281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1cfdb-32c6-4d92-8c1d-588860eb5686 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:28.414564 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1cfdb-32c6-4d92-8c1d-588860eb5686 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:28.414839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1cfdb-32c6-4d92-8c1d-588860eb5686 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:28.415605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b5c6414b-b310-4d10-905f-c5d3353546e5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 18:49:28.415605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:28.415605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:28.415605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:28.415605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:28.415605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:28.415605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:28.415605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:28.415605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:28.415605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:28.415605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:28.415605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:28.415605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:28.415605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:28.415605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:28.415605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:28.416669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:28.416669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:28.416669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:28.416669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:28.416669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:28.416669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:28.416669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:28.416669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:28.416669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:28.416669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:28.416669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:28.416669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:28.416669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:28.416669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:28.416669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:28.416669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:28.416669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:28.416669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:28.417563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:28.417563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:28.417563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:28.417563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:28.417563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:28.417563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:28.417563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:28.417563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:28.417563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 18:49:28.417563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:28.417563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 241/498] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:28.417563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6414b-b310-4d10-905f-c5d3353546e5 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:28.426398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d1cfdb-32c6-4d92-8c1d-588860eb5686 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:28.427885 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d1cfdb-32c6-4d92-8c1d-588860eb5686 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:28.443458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-58d1cfdb-32c6-4d92-8c1d-588860eb5686 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 18:49:28.443458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:28.443458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:28.443458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:28.443458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:28.443458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:28.443458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:28.443458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:28.443458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:28.443458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:28.443458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:28.443458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:28.443458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:28.443458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:28.443458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:28.443458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:28.444347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:28.444347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:28.444347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:28.444347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:28.444347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:28.444347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:28.444347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:28.444347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:28.444347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:28.444347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:28.444347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:28.444347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:28.444347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:28.444347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:28.444347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:28.444347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:28.444347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:28.444347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:28.445192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:28.445192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:28.445192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:28.445192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:28.445192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:28.445192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:28.445192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:28.445192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:28.445192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 18:49:28.445192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:28.445192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 258/499] 10.222.0.69 () {56 vars in 1093 bytes} [Thu Jul 14 18:49:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:28.446285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69570099-488b-47b2-a379-83989349e85b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:28.446549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69570099-488b-47b2-a379-83989349e85b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:28.446903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69570099-488b-47b2-a379-83989349e85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:28.464194 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-69570099-488b-47b2-a379-83989349e85b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:28.465398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-69570099-488b-47b2-a379-83989349e85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:28.489879 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:28.524642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6414b-b310-4d10-905f-c5d3353546e5 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:28.531935 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 259/500] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:28.533837 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8905a49-b312-4888-8a87-c07856369b66 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:28.534492 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8905a49-b312-4888-8a87-c07856369b66 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:28.535357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8905a49-b312-4888-8a87-c07856369b66 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:28.537118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 260/501] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:28.537681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8905a49-b312-4888-8a87-c07856369b66 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:28.564797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc18663-e64f-458e-ba2e-40a142b3ba46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:28.565577 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc18663-e64f-458e-ba2e-40a142b3ba46 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:28.565867 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc18663-e64f-458e-ba2e-40a142b3ba46 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:28.566149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc18663-e64f-458e-ba2e-40a142b3ba46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:28.576249 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc18663-e64f-458e-ba2e-40a142b3ba46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:28.577745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc18663-e64f-458e-ba2e-40a142b3ba46 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:28.595504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-1bc18663-e64f-458e-ba2e-40a142b3ba46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:28.595504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:28.595504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:28.595504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:28.595504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:28.595504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:28.595504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:28.595504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:28.595504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:28.595504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:28.595504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:28.595504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:28.595504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:28.595504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:28.595504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:28.596522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:28.596522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:28.596522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:28.596522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:28.596522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:28.596522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:28.596522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:28.596522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:28.596522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:28.596522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:28.596522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:28.596522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:28.596522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:28.596522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:28.596522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:28.596522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:28.596522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:28.598527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:28.598527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:28.598527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:28.598527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:28.598527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 242/502] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:28.601524 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc18663-e64f-458e-ba2e-40a142b3ba46 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:28.637323 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8905a49-b312-4888-8a87-c07856369b66 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:28.642999 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:28.671490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018fe4a2-a959-4f4d-8b96-94d3f9a15f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:28.672087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fe4a2-a959-4f4d-8b96-94d3f9a15f8e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:28.672370 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fe4a2-a959-4f4d-8b96-94d3f9a15f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:28.672640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fe4a2-a959-4f4d-8b96-94d3f9a15f8e None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:28.673465 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018fe4a2-a959-4f4d-8b96-94d3f9a15f8e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:28.673771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018fe4a2-a959-4f4d-8b96-94d3f9a15f8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:28.675250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018fe4a2-a959-4f4d-8b96-94d3f9a15f8e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:28.702276 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc18663-e64f-458e-ba2e-40a142b3ba46 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:28.710367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 261/503] 10.222.0.69 () {56 vars in 1137 bytes} [Thu Jul 14 18:49:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:28.711706 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95c95c-d570-465a-8613-bece966440d4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:28.712003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95c95c-d570-465a-8613-bece966440d4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:28.712280 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95c95c-d570-465a-8613-bece966440d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:28.728499 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ae95c95c-d570-465a-8613-bece966440d4 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:28.729677 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ae95c95c-d570-465a-8613-bece966440d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:28.740852 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98adba-64c8-47e5-a99a-7fda4d04d2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:28.741525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98adba-64c8-47e5-a99a-7fda4d04d2be None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:28.741812 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98adba-64c8-47e5-a99a-7fda4d04d2be None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:28.742091 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98adba-64c8-47e5-a99a-7fda4d04d2be None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:28.742390 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:28.743187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98adba-64c8-47e5-a99a-7fda4d04d2be None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:28.743320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98adba-64c8-47e5-a99a-7fda4d04d2be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:28.744694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98adba-64c8-47e5-a99a-7fda4d04d2be None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:28.782373 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 243/504] 10.222.0.69 () {56 vars in 1133 bytes} [Thu Jul 14 18:49:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:28.783418 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98adba-64c8-47e5-a99a-7fda4d04d2be None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:28.807188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 262/505] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:28.807188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95c95c-d570-465a-8613-bece966440d4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:28.882985 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:28.886901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98adba-64c8-47e5-a99a-7fda4d04d2be None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:28.908106 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95c95c-d570-465a-8613-bece966440d4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:28.920448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f206c3e-6ea5-4960-a6a8-dedbd5c60c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:28.921236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f206c3e-6ea5-4960-a6a8-dedbd5c60c3a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:28.921567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f206c3e-6ea5-4960-a6a8-dedbd5c60c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:28.921796 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f206c3e-6ea5-4960-a6a8-dedbd5c60c3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:28.931898 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f206c3e-6ea5-4960-a6a8-dedbd5c60c3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:28.939569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-2f206c3e-6ea5-4960-a6a8-dedbd5c60c3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:28.939569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:28.939569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:28.939569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:28.939569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:28.939569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:28.939569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:28.939569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:28.939569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:28.939569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:28.939569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:28.939569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:28.939569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:28.939569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:28.939569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:28.939569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:28.940611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:28.940611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:28.940611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:28.940611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:28.940611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:28.940611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:28.940611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:28.940611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:28.940611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:28.940611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:28.940611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:28.940611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:28.940611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:28.940611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:28.940611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:28.940611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:28.942094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:28.942094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:28.942094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:28.942094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:28.942094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:28.942094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:28.942094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:28.942094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:28.942094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:28.942094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:28.942094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:28.942094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:28.942978 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6dc7bf-a347-4115-bd37-c85cb4230e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:28.943678 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6dc7bf-a347-4115-bd37-c85cb4230e3b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:28.943964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6dc7bf-a347-4115-bd37-c85cb4230e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:28.944241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6dc7bf-a347-4115-bd37-c85cb4230e3b None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:28.944958 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6dc7bf-a347-4115-bd37-c85cb4230e3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:28.945265 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6dc7bf-a347-4115-bd37-c85cb4230e3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:28.946116 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 244/506] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:28.946660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6dc7bf-a347-4115-bd37-c85cb4230e3b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:28.946795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f206c3e-6ea5-4960-a6a8-dedbd5c60c3a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:28.958669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 263/507] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:28.959573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6dc7bf-a347-4115-bd37-c85cb4230e3b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:28.970084 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:29.049666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f206c3e-6ea5-4960-a6a8-dedbd5c60c3a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:29.061256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6dc7bf-a347-4115-bd37-c85cb4230e3b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:29.083138 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf397f7-3ee8-410b-a417-81ab389a1464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:29.083886 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf397f7-3ee8-410b-a417-81ab389a1464 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:29.084175 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf397f7-3ee8-410b-a417-81ab389a1464 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:29.084454 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf397f7-3ee8-410b-a417-81ab389a1464 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:29.094930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc43944-48dd-42eb-a8be-3bd0fc7f171b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:29.095586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc43944-48dd-42eb-a8be-3bd0fc7f171b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:29.095881 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc43944-48dd-42eb-a8be-3bd0fc7f171b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:29.096168 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc43944-48dd-42eb-a8be-3bd0fc7f171b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:29.110256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:29.151120 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf397f7-3ee8-410b-a417-81ab389a1464 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:29.152868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf397f7-3ee8-410b-a417-81ab389a1464 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:29.164179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc43944-48dd-42eb-a8be-3bd0fc7f171b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:29.165654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc43944-48dd-42eb-a8be-3bd0fc7f171b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:29.175732 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:29.253428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 245/508] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:29.255191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf397f7-3ee8-410b-a417-81ab389a1464 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:29.273680 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 264/509] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:29.274911 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc43944-48dd-42eb-a8be-3bd0fc7f171b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:29.320089 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:29.362084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf397f7-3ee8-410b-a417-81ab389a1464 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:29.371402 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc43944-48dd-42eb-a8be-3bd0fc7f171b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:29.380861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:29.395554 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c9ba4-4709-4457-9e64-be2a18605783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:29.396352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c9ba4-4709-4457-9e64-be2a18605783 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:29.396637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c9ba4-4709-4457-9e64-be2a18605783 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:29.396948 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c9ba4-4709-4457-9e64-be2a18605783 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:29.404550 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc34d2d-cdc5-47ce-89e0-d6985bb72fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:29.405206 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc34d2d-cdc5-47ce-89e0-d6985bb72fdb None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:29.405487 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc34d2d-cdc5-47ce-89e0-d6985bb72fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:29.405765 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc34d2d-cdc5-47ce-89e0-d6985bb72fdb None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:29.406485 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc34d2d-cdc5-47ce-89e0-d6985bb72fdb None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:29.406788 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc34d2d-cdc5-47ce-89e0-d6985bb72fdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:29.408139 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c9ba4-4709-4457-9e64-be2a18605783 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:29.408483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc34d2d-cdc5-47ce-89e0-d6985bb72fdb None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:29.409653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c9ba4-4709-4457-9e64-be2a18605783 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:29.416673 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 265/510] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:49:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:29.417442 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc34d2d-cdc5-47ce-89e0-d6985bb72fdb None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:29.427794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-6c0c9ba4-4709-4457-9e64-be2a18605783 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:29.427794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:29.427794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:29.427794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:29.427794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:29.427794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:29.427794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:29.427794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:29.427794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:29.427794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:29.427794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:29.427794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:29.427794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:29.427794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:29.427794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:29.428616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:29.428616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:29.428616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:29.428616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:29.428616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:29.428616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:29.428616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:29.428616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:29.428616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:29.428616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:29.428616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:29.428616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:29.428616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:29.428616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:29.428616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:29.428616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:29.428616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:29.429451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:29.429451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:29.429451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:29.429451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:29.429451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 246/511] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:29.429719 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c9ba4-4709-4457-9e64-be2a18605783 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:29.516603 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc34d2d-cdc5-47ce-89e0-d6985bb72fdb None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:29.533371 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c9ba4-4709-4457-9e64-be2a18605783 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:29.544969 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:29.550198 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cfa00-8a35-43f8-a16b-00b26fd39926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:29.551613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cfa00-8a35-43f8-a16b-00b26fd39926 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:29.557440 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cfa00-8a35-43f8-a16b-00b26fd39926 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:29.557440 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cfa00-8a35-43f8-a16b-00b26fd39926 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:29.559490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cfa00-8a35-43f8-a16b-00b26fd39926 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:29.560936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cfa00-8a35-43f8-a16b-00b26fd39926 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:29.568005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bda02-065a-4bda-b653-6cda6400324c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:29.568791 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bda02-065a-4bda-b653-6cda6400324c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:29.569078 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bda02-065a-4bda-b653-6cda6400324c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:29.569350 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bda02-065a-4bda-b653-6cda6400324c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:29.574861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-090cfa00-8a35-43f8-a16b-00b26fd39926 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 18:49:29.574861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:29.574861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:29.574861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:29.574861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:29.574861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:29.574861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:29.574861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:29.574861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:29.574861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:29.574861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:29.574861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:29.574861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:29.574861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:29.574861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:29.574861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:29.575888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:29.575888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:29.575888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:29.575888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:29.575888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:29.575888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:29.575888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:29.575888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:29.575888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:29.575888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:29.575888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:29.575888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:29.575888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:29.575888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:29.575888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:29.575888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:29.575888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:29.575888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:29.581504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:29.581504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:29.581504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:29.581504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:29.581504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:29.581504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:29.581504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:29.581504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:29.581504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 18:49:29.581504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:29.581504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 266/512] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:29.581504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cfa00-8a35-43f8-a16b-00b26fd39926 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:29.582277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bda02-065a-4bda-b653-6cda6400324c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:29.582277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bda02-065a-4bda-b653-6cda6400324c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:29.600667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-670bda02-065a-4bda-b653-6cda6400324c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:29.600667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:29.600667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:29.600667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:29.600667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:29.600667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:29.600667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:29.600667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:29.600667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:29.600667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:29.600667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:29.600667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:29.600667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:29.600667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:29.600667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:29.601651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:29.601651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:29.601651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:29.601651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:29.601651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:29.601651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:29.601651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:29.601651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:29.601651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:29.601651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:29.601651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:29.601651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:29.601651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:29.601651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:29.601651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:29.601651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:29.601651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:29.602654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:29.602654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:29.602654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:29.602654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:29.602654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 247/513] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:29.606986 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bda02-065a-4bda-b653-6cda6400324c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:29.611412 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:29.679890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cfa00-8a35-43f8-a16b-00b26fd39926 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:29.704589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bda02-065a-4bda-b653-6cda6400324c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:29.712455 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524b7c4-aa94-415f-b393-8d8d70b645c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:29.713248 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524b7c4-aa94-415f-b393-8d8d70b645c4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:29.713532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524b7c4-aa94-415f-b393-8d8d70b645c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:29.713807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524b7c4-aa94-415f-b393-8d8d70b645c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:29.724187 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524b7c4-aa94-415f-b393-8d8d70b645c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:29.725661 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524b7c4-aa94-415f-b393-8d8d70b645c4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:29.738944 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031721f6-742c-4813-9dc5-3dc802a11c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:29.739752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031721f6-742c-4813-9dc5-3dc802a11c64 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:29.740036 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031721f6-742c-4813-9dc5-3dc802a11c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:29.740313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031721f6-742c-4813-9dc5-3dc802a11c64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:29.743321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-a524b7c4-aa94-415f-b393-8d8d70b645c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:29.743321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:29.743321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:29.743321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:29.743321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:29.743321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:29.743321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:29.743321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:29.743321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:29.743321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:29.743321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:29.743321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:29.743321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:29.743321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:29.743321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:29.744311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:29.744311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:29.744311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:29.744311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:29.744311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:29.744311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:29.744311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:29.744311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:29.744311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:29.744311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:29.744311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:29.744311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:29.744311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:29.744311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:29.744311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:29.744311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:29.744311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:29.745321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:29.745321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:29.745321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:29.745321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:29.745321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 267/514] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:29.745321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524b7c4-aa94-415f-b393-8d8d70b645c4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:29.752882 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031721f6-742c-4813-9dc5-3dc802a11c64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:29.754376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031721f6-742c-4813-9dc5-3dc802a11c64 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:29.771390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-031721f6-742c-4813-9dc5-3dc802a11c64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:29.771390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:29.771390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:29.771390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:29.771390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:29.771390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:29.771390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:29.771390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:29.771390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:29.771390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:29.771390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:29.771390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:29.771390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:29.771390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:29.771390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:29.772417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:29.772417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:29.772417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:29.772417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:29.772417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:29.772417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:29.772417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:29.772417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:29.772417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:29.772417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:29.772417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:29.772417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:29.772417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:29.772417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:29.772417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:29.772417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:29.772417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:29.773433 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:29.773433 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:29.773433 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:29.773433 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:29.777251 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 248/515] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:29.778071 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031721f6-742c-4813-9dc5-3dc802a11c64 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:29.794515 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:29.827712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:29.845360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524b7c4-aa94-415f-b393-8d8d70b645c4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:29.877598 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031721f6-742c-4813-9dc5-3dc802a11c64 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:29.880807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb812f-903d-4afe-9095-dd7ffcc80f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:29.881466 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb812f-903d-4afe-9095-dd7ffcc80f8e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:29.881753 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb812f-903d-4afe-9095-dd7ffcc80f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:29.882034 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb812f-903d-4afe-9095-dd7ffcc80f8e None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:29.882753 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb812f-903d-4afe-9095-dd7ffcc80f8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:29.883064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb812f-903d-4afe-9095-dd7ffcc80f8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:29.884554 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb812f-903d-4afe-9095-dd7ffcc80f8e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:29.895619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 268/516] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:29.896407 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb812f-903d-4afe-9095-dd7ffcc80f8e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:29.911752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd768b7-254f-4664-8cf7-b4683e309607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:29.912449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd768b7-254f-4664-8cf7-b4683e309607 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:29.912739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd768b7-254f-4664-8cf7-b4683e309607 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:29.913019 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd768b7-254f-4664-8cf7-b4683e309607 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:29.913848 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd768b7-254f-4664-8cf7-b4683e309607 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:29.914160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd768b7-254f-4664-8cf7-b4683e309607 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:29.915631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd768b7-254f-4664-8cf7-b4683e309607 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:29.932978 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 249/517] 10.222.0.69 () {56 vars in 1137 bytes} [Thu Jul 14 18:49:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:29.933700 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd768b7-254f-4664-8cf7-b4683e309607 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:30.000420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb812f-903d-4afe-9095-dd7ffcc80f8e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:30.002335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:30.032204 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd768b7-254f-4664-8cf7-b4683e309607 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:30.035706 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41299758-d4c3-4d5e-81a9-2c1edfd2f832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:30.036372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41299758-d4c3-4d5e-81a9-2c1edfd2f832 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:30.036667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41299758-d4c3-4d5e-81a9-2c1edfd2f832 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:30.036931 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41299758-d4c3-4d5e-81a9-2c1edfd2f832 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:30.037640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41299758-d4c3-4d5e-81a9-2c1edfd2f832 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:30.037944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41299758-d4c3-4d5e-81a9-2c1edfd2f832 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:30.039352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41299758-d4c3-4d5e-81a9-2c1edfd2f832 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:30.050718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 269/518] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:30.051532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41299758-d4c3-4d5e-81a9-2c1edfd2f832 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:30.054145 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:30.065962 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb205614-3b85-44e8-9ba3-27a841dc18cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:30.066647 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb205614-3b85-44e8-9ba3-27a841dc18cd None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:30.066907 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb205614-3b85-44e8-9ba3-27a841dc18cd None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:30.067205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb205614-3b85-44e8-9ba3-27a841dc18cd None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:30.067933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb205614-3b85-44e8-9ba3-27a841dc18cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:30.068239 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb205614-3b85-44e8-9ba3-27a841dc18cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:30.069623 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb205614-3b85-44e8-9ba3-27a841dc18cd None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:30.081990 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 250/519] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:30.082798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb205614-3b85-44e8-9ba3-27a841dc18cd None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:30.153480 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41299758-d4c3-4d5e-81a9-2c1edfd2f832 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:30.186996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1dedf-a29a-4e3b-ba8c-03f60f4185b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:30.187604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1dedf-a29a-4e3b-ba8c-03f60f4185b3 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:30.187891 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1dedf-a29a-4e3b-ba8c-03f60f4185b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:30.188168 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1dedf-a29a-4e3b-ba8c-03f60f4185b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:30.188877 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1dedf-a29a-4e3b-ba8c-03f60f4185b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:30.189183 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1dedf-a29a-4e3b-ba8c-03f60f4185b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:30.190550 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1dedf-a29a-4e3b-ba8c-03f60f4185b3 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:30.201441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 270/520] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:30.202232 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1dedf-a29a-4e3b-ba8c-03f60f4185b3 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:30.202621 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb205614-3b85-44e8-9ba3-27a841dc18cd None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:30.233273 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:30.238208 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e949c4-9838-4a86-b9ea-514ee840d258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:30.238883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e949c4-9838-4a86-b9ea-514ee840d258 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:30.239192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e949c4-9838-4a86-b9ea-514ee840d258 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:30.239478 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e949c4-9838-4a86-b9ea-514ee840d258 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:30.240180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e949c4-9838-4a86-b9ea-514ee840d258 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:30.241689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e949c4-9838-4a86-b9ea-514ee840d258 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:30.262663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:30.300243 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1dedf-a29a-4e3b-ba8c-03f60f4185b3 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:30.332313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f9fca-4b96-495c-925b-85831ca23da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:30.332939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f9fca-4b96-495c-925b-85831ca23da6 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:30.333259 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f9fca-4b96-495c-925b-85831ca23da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:30.333535 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f9fca-4b96-495c-925b-85831ca23da6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:30.339618 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 251/521] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 14 18:49:30 2022] POST /identity/v3/users => generated 247 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:49:30.340437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e949c4-9838-4a86-b9ea-514ee840d258 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:30.415509 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f9fca-4b96-495c-925b-85831ca23da6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:30.417010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f9fca-4b96-495c-925b-85831ca23da6 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:30.466635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e949c4-9838-4a86-b9ea-514ee840d258 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:30.484299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:30.514589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9438d-5761-4511-8bc7-e203697f2671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:30.515427 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9438d-5761-4511-8bc7-e203697f2671 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:30.515724 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9438d-5761-4511-8bc7-e203697f2671 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:30.516002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9438d-5761-4511-8bc7-e203697f2671 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:30.524472 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9438d-5761-4511-8bc7-e203697f2671 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:30.525944 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9438d-5761-4511-8bc7-e203697f2671 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:30.532558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 271/522] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:30.533415 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f9fca-4b96-495c-925b-85831ca23da6 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:30.543383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-2bc9438d-5761-4511-8bc7-e203697f2671 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:30.543383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:30.543383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:30.543383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:30.543383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:30.543383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:30.543383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:30.543383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:30.543383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:30.543383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:30.543383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:30.543383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:30.543383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:30.543383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:30.543383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:30.544395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:30.544395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:30.544395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:30.544395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:30.544395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:30.544395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:30.544395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:30.544395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:30.544395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:30.544395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:30.544395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:30.544395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:30.544395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:30.544395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:30.544395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:30.544395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:30.544395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:30.545233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:30.545233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:30.545233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:30.545233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:30.545233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:30.545233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:30.545233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:30.545233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:30.545233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 252/523] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:30.546131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9438d-5761-4511-8bc7-e203697f2671 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:30.601109 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:30.649296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f9fca-4b96-495c-925b-85831ca23da6 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:30.654791 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9438d-5761-4511-8bc7-e203697f2671 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:30.687408 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8f098-de5c-4569-a714-33958d596d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:30.688212 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8f098-de5c-4569-a714-33958d596d57 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:30.688501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8f098-de5c-4569-a714-33958d596d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:30.688782 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8f098-de5c-4569-a714-33958d596d57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:30.691933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126d0f1-a94a-4310-a063-df37143d5048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:30.692739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126d0f1-a94a-4310-a063-df37143d5048 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:30.693029 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126d0f1-a94a-4310-a063-df37143d5048 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:30.693308 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126d0f1-a94a-4310-a063-df37143d5048 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:30.695850 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8f098-de5c-4569-a714-33958d596d57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:30.697322 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8f098-de5c-4569-a714-33958d596d57 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:30.701374 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126d0f1-a94a-4310-a063-df37143d5048 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:30.702815 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126d0f1-a94a-4310-a063-df37143d5048 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:30.712519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-b9f8f098-de5c-4569-a714-33958d596d57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:30.712519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:30.712519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:30.712519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:30.712519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:30.712519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:30.712519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:30.712519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:30.712519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:30.712519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:30.712519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:30.712519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:30.712519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:30.712519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:30.712519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:30.713425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:30.713425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:30.713425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:30.713425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:30.713425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:30.713425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:30.713425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:30.713425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:30.713425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:30.713425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:30.713425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:30.713425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:30.713425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:30.713425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:30.713425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:30.713425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:30.713425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:30.714278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:30.714278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:30.714278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:30.714278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:30.714278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:30.714278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:30.714278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:30.714278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:30.714278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 272/524] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:30.715215 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8f098-de5c-4569-a714-33958d596d57 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:30.723364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-a126d0f1-a94a-4310-a063-df37143d5048 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 18:49:30.723364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:30.723364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:30.723364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:30.723364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:30.723364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:30.723364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:30.723364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:30.723364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:30.723364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:30.723364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:30.723364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:30.723364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:30.723364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:30.723364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:30.723364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:30.724243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:30.724243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:30.724243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:30.724243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:30.724243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:30.724243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:30.724243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:30.724243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:30.724243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:30.724243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:30.724243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:30.724243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:30.724243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:30.724243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:30.724243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:30.724243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:30.724243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:30.724243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:30.725147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:30.725147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:30.725147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:30.725147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:30.725147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:30.725147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:30.725147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:30.725147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:30.725147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 18:49:30.725147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:30.727465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 253/525] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:30.728994 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126d0f1-a94a-4310-a063-df37143d5048 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:30.742701 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:30.822487 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8f098-de5c-4569-a714-33958d596d57 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:30.840199 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126d0f1-a94a-4310-a063-df37143d5048 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:30.849861 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:30.862222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2d64c-25b0-4290-83cf-e8765bb10b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:30.863030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2d64c-25b0-4290-83cf-e8765bb10b98 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:30.863351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2d64c-25b0-4290-83cf-e8765bb10b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:30.863631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2d64c-25b0-4290-83cf-e8765bb10b98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:30.874716 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc2d64c-25b0-4290-83cf-e8765bb10b98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:30.876223 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc2d64c-25b0-4290-83cf-e8765bb10b98 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:30.876892 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e29d5-98cf-43a1-a218-db21ac2a29a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:30.877572 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e29d5-98cf-43a1-a218-db21ac2a29a5 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:30.878063 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e29d5-98cf-43a1-a218-db21ac2a29a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:30.878395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e29d5-98cf-43a1-a218-db21ac2a29a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:30.879285 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e29d5-98cf-43a1-a218-db21ac2a29a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:30.879839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e29d5-98cf-43a1-a218-db21ac2a29a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:30.881562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e29d5-98cf-43a1-a218-db21ac2a29a5 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:30.896288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-abc2d64c-25b0-4290-83cf-e8765bb10b98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:30.896288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:30.896288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:30.896288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:30.896288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:30.896288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:30.896288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:30.896288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:30.896288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:30.896288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:30.896288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:30.896288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:30.896288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:30.896288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:30.896288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:30.897125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:30.897125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:30.897125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:30.897125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:30.897125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:30.897125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:30.897125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:30.897125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:30.897125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:30.897125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:30.897125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:30.897125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:30.897125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:30.897125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:30.897125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:30.897125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:30.897125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:30.897940 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:30.897940 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:30.897940 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:30.897940 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:30.897940 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 273/526] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:30.898622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2d64c-25b0-4290-83cf-e8765bb10b98 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:30.910787 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 254/527] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:30.910787 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e29d5-98cf-43a1-a218-db21ac2a29a5 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:30.978768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:31.008224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2d64c-25b0-4290-83cf-e8765bb10b98 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:31.018715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e29d5-98cf-43a1-a218-db21ac2a29a5 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:31.048860 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ff87b-89a4-4684-afda-c189733a1d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:31.049658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ff87b-89a4-4684-afda-c189733a1d2d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:31.050178 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ff87b-89a4-4684-afda-c189733a1d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:31.050286 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ff87b-89a4-4684-afda-c189733a1d2d None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:31.051190 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ff87b-89a4-4684-afda-c189733a1d2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:31.051439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ff87b-89a4-4684-afda-c189733a1d2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:31.052881 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ff87b-89a4-4684-afda-c189733a1d2d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:31.055329 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e1e65-e103-4d16-a4d8-332b66a6a686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:31.056004 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e1e65-e103-4d16-a4d8-332b66a6a686 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:31.056293 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e1e65-e103-4d16-a4d8-332b66a6a686 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:31.056963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e1e65-e103-4d16-a4d8-332b66a6a686 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:31.057794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e1e65-e103-4d16-a4d8-332b66a6a686 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:31.058107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e1e65-e103-4d16-a4d8-332b66a6a686 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:31.059574 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e1e65-e103-4d16-a4d8-332b66a6a686 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:31.075490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 274/528] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:31.075490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ff87b-89a4-4684-afda-c189733a1d2d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:31.079798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 255/529] 10.222.0.69 () {56 vars in 1139 bytes} [Thu Jul 14 18:49:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:31.084854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e1e65-e103-4d16-a4d8-332b66a6a686 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:31.129399 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:31.189106 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ff87b-89a4-4684-afda-c189733a1d2d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:31.206298 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e1e65-e103-4d16-a4d8-332b66a6a686 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:31.226301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12501d37-9ea9-411e-8d66-bb5f32ace6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:31.226964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12501d37-9ea9-411e-8d66-bb5f32ace6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:31.227275 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12501d37-9ea9-411e-8d66-bb5f32ace6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:31.227555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12501d37-9ea9-411e-8d66-bb5f32ace6c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:31.228263 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12501d37-9ea9-411e-8d66-bb5f32ace6c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:31.228568 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12501d37-9ea9-411e-8d66-bb5f32ace6c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:31.230009 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12501d37-9ea9-411e-8d66-bb5f32ace6c8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:31.231646 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:31.245617 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 275/530] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:31.245617 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12501d37-9ea9-411e-8d66-bb5f32ace6c8 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:31.245870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50772710-4ee9-4a12-aeb9-5824b757ccb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:31.245870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50772710-4ee9-4a12-aeb9-5824b757ccb6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:31.245870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50772710-4ee9-4a12-aeb9-5824b757ccb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:31.245870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50772710-4ee9-4a12-aeb9-5824b757ccb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:31.246176 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50772710-4ee9-4a12-aeb9-5824b757ccb6 None admin] RBAC: Adding query filter params (role.id=24846e82841b4b05b4d9e9d193a98868, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=31b09d72314349c6be0f906484895608) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:31.246176 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50772710-4ee9-4a12-aeb9-5824b757ccb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:31.246327 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50772710-4ee9-4a12-aeb9-5824b757ccb6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:31.251818 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 256/531] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:49:31 2022] GET /identity/v3/role_assignments?user.id=31b09d72314349c6be0f906484895608&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=24846e82841b4b05b4d9e9d193a98868 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:31.252725 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50772710-4ee9-4a12-aeb9-5824b757ccb6 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:31.359638 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12501d37-9ea9-411e-8d66-bb5f32ace6c8 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:31.399811 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:31.407426 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d7519-7bfd-4e9f-916c-8bc9599eebbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:31.408369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d7519-7bfd-4e9f-916c-8bc9599eebbd None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:31.408896 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d7519-7bfd-4e9f-916c-8bc9599eebbd None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:31.409173 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d7519-7bfd-4e9f-916c-8bc9599eebbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:31.410123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d7519-7bfd-4e9f-916c-8bc9599eebbd None admin] RBAC: Adding query filter params (role.id=24846e82841b4b05b4d9e9d193a98868, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=8d27d6df5cb64697adebd961dfbec324) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:31.410438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d7519-7bfd-4e9f-916c-8bc9599eebbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:31.411891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50772710-4ee9-4a12-aeb9-5824b757ccb6 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:31.412003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d7519-7bfd-4e9f-916c-8bc9599eebbd None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:31.417795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 276/532] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:49:31 2022] GET /identity/v3/role_assignments?user.id=8d27d6df5cb64697adebd961dfbec324&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=24846e82841b4b05b4d9e9d193a98868 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:31.418737 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d7519-7bfd-4e9f-916c-8bc9599eebbd None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:31.455333 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c80a2-00ad-4e91-9775-a39eaf3d4c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:31.456190 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c80a2-00ad-4e91-9775-a39eaf3d4c20 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:31.456568 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c80a2-00ad-4e91-9775-a39eaf3d4c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:31.456924 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c80a2-00ad-4e91-9775-a39eaf3d4c20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:31.492114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:31.547886 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c80a2-00ad-4e91-9775-a39eaf3d4c20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:31.549776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c80a2-00ad-4e91-9775-a39eaf3d4c20 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:31.571376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d7519-7bfd-4e9f-916c-8bc9599eebbd None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:31.656821 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342a0f9-2f4e-4f76-8867-86a941f44a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:31.657740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342a0f9-2f4e-4f76-8867-86a941f44a0a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:31.658468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342a0f9-2f4e-4f76-8867-86a941f44a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:31.658760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342a0f9-2f4e-4f76-8867-86a941f44a0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:31.670247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342a0f9-2f4e-4f76-8867-86a941f44a0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:31.672250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:31.672987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342a0f9-2f4e-4f76-8867-86a941f44a0a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:31.694693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 257/533] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:31.700239 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c80a2-00ad-4e91-9775-a39eaf3d4c20 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:31.712937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c342a0f9-2f4e-4f76-8867-86a941f44a0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:31.712937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:31.712937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:31.712937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:31.712937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:31.712937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:31.712937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:31.712937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:31.712937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:31.712937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:31.712937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:31.712937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:31.712937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:31.712937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:31.712937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:31.713823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:31.713823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:31.713823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:31.713823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:31.713823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:31.713823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:31.713823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:31.713823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:31.713823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:31.713823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:31.713823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:31.713823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:31.713823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:31.713823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:31.713823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:31.713823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:31.713823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:31.714659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:31.714659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:31.714659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:31.714659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:31.714659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:31.714659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:31.714659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:31.714659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:31.714659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 277/534] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:31.719880 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342a0f9-2f4e-4f76-8867-86a941f44a0a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:31.883019 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c80a2-00ad-4e91-9775-a39eaf3d4c20 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:31.918951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342a0f9-2f4e-4f76-8867-86a941f44a0a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:31.952711 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:31.954013 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f93594-29df-4f3d-8967-c623ba57ead1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:31.954815 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f93594-29df-4f3d-8967-c623ba57ead1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:31.955098 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f93594-29df-4f3d-8967-c623ba57ead1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:31.955393 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f93594-29df-4f3d-8967-c623ba57ead1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:31.970713 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f93594-29df-4f3d-8967-c623ba57ead1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:31.972953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f93594-29df-4f3d-8967-c623ba57ead1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:31.987045 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446965df-0d54-478f-85b9-1f350214abe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:31.988307 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446965df-0d54-478f-85b9-1f350214abe9 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:31.988594 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446965df-0d54-478f-85b9-1f350214abe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:31.988933 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446965df-0d54-478f-85b9-1f350214abe9 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:31.989728 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446965df-0d54-478f-85b9-1f350214abe9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:31.991598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446965df-0d54-478f-85b9-1f350214abe9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:31.993054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446965df-0d54-478f-85b9-1f350214abe9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:32.008526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-38f93594-29df-4f3d-8967-c623ba57ead1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:32.008526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:32.008526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:32.008526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:32.008526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:32.008526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:32.008526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:32.008526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:32.008526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:32.008526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:32.008526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:32.008526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:32.008526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:32.008526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:32.008526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:32.009508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:32.009508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:32.009508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:32.009508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:32.009508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:32.009508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:32.009508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:32.009508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:32.009508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:32.009508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:32.009508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:32.009508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:32.009508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:32.009508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:32.009508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:32.009508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:32.009508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:32.010547 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:32.010547 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:32.010547 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:32.010547 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:32.015189 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 258/535] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:32.015913 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f93594-29df-4f3d-8967-c623ba57ead1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:32.016654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 278/536] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:32.022998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446965df-0d54-478f-85b9-1f350214abe9 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:32.101080 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:32.161577 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f93594-29df-4f3d-8967-c623ba57ead1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:32.197496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b0e82-2cb8-495e-b5fa-d10b710250a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:32.198152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b0e82-2cb8-495e-b5fa-d10b710250a7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:32.198491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b0e82-2cb8-495e-b5fa-d10b710250a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:32.198776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b0e82-2cb8-495e-b5fa-d10b710250a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:32.199536 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b0e82-2cb8-495e-b5fa-d10b710250a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:32.199848 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b0e82-2cb8-495e-b5fa-d10b710250a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:32.201226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b0e82-2cb8-495e-b5fa-d10b710250a7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:32.214469 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 259/537] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:32.220343 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b0e82-2cb8-495e-b5fa-d10b710250a7 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:32.247338 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446965df-0d54-478f-85b9-1f350214abe9 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:32.318215 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47373252-6da8-4b79-a08c-6b0c0cfa5cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:32.318934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47373252-6da8-4b79-a08c-6b0c0cfa5cbe None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:32.319249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47373252-6da8-4b79-a08c-6b0c0cfa5cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:32.319529 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47373252-6da8-4b79-a08c-6b0c0cfa5cbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:32.320420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47373252-6da8-4b79-a08c-6b0c0cfa5cbe None admin] RBAC: Adding query filter params (role.id=24846e82841b4b05b4d9e9d193a98868, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=e5d76eae62b54521a95f36284118fada) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:32.320675 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47373252-6da8-4b79-a08c-6b0c0cfa5cbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:32.324211 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47373252-6da8-4b79-a08c-6b0c0cfa5cbe None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:32.335574 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 279/538] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:49:32 2022] GET /identity/v3/role_assignments?user.id=e5d76eae62b54521a95f36284118fada&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=24846e82841b4b05b4d9e9d193a98868 => generated 259 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:32.381378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b0e82-2cb8-495e-b5fa-d10b710250a7 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:32.382896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:32.419962 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938aa39e-8d88-4ad3-8e07-5e945114688c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:32.420698 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938aa39e-8d88-4ad3-8e07-5e945114688c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:32.420985 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938aa39e-8d88-4ad3-8e07-5e945114688c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:32.421261 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938aa39e-8d88-4ad3-8e07-5e945114688c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:32.429496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938aa39e-8d88-4ad3-8e07-5e945114688c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:32.431086 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938aa39e-8d88-4ad3-8e07-5e945114688c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:32.446062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-938aa39e-8d88-4ad3-8e07-5e945114688c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:32.446062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:32.446062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:32.446062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:32.446062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:32.446062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:32.446062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:32.446062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:32.446062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:32.446062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:32.446062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:32.446062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:32.446062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:32.446062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:32.446062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:32.446915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:32.446915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:32.446915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:32.446915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:32.446915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:32.446915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:32.446915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:32.446915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:32.446915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:32.446915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:32.446915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:32.446915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:32.446915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:32.446915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:32.446915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:32.446915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:32.446915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:32.447769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:32.447769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:32.447769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:32.447769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:32.447769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:32.447769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:32.447769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:32.447769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:32.447769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 260/539] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:32.457143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47373252-6da8-4b79-a08c-6b0c0cfa5cbe None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:32.507695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bd6dc-3fdf-493b-aa0f-7ea79e8f8ec6 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:32.508029 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bd6dc-3fdf-493b-aa0f-7ea79e8f8ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:32.508259 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bd6dc-3fdf-493b-aa0f-7ea79e8f8ec6 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:32.509919 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 261/540] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:49:32.515398 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9b1d1-ab9a-42b8-a3b0-c1b33daa7b2a None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:32.515690 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9b1d1-ab9a-42b8-a3b0-c1b33daa7b2a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:32.515976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9b1d1-ab9a-42b8-a3b0-c1b33daa7b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:32.533879 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:32.535732 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7dc9b1d1-ab9a-42b8-a3b0-c1b33daa7b2a None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:32.536976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7dc9b1d1-ab9a-42b8-a3b0-c1b33daa7b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:32.607699 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47373252-6da8-4b79-a08c-6b0c0cfa5cbe None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:32.613575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 262/541] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:49:32.637997 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9b1d1-ab9a-42b8-a3b0-c1b33daa7b2a None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:32.646680 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246f19d-8090-4c2b-a9e4-f7599d103fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:32.647760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246f19d-8090-4c2b-a9e4-f7599d103fee None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:32.648274 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246f19d-8090-4c2b-a9e4-f7599d103fee None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:32.648742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246f19d-8090-4c2b-a9e4-f7599d103fee None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:32.649734 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246f19d-8090-4c2b-a9e4-f7599d103fee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:32.650338 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246f19d-8090-4c2b-a9e4-f7599d103fee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:32.652399 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246f19d-8090-4c2b-a9e4-f7599d103fee None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:32.666323 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 280/542] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:32.672434 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246f19d-8090-4c2b-a9e4-f7599d103fee None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:32.743131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:32.754759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9b1d1-ab9a-42b8-a3b0-c1b33daa7b2a None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:32.805024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf8e78-0fae-4277-82dd-1c32ea597305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:32.807403 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf8e78-0fae-4277-82dd-1c32ea597305 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:32.809208 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf8e78-0fae-4277-82dd-1c32ea597305 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:32.810460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf8e78-0fae-4277-82dd-1c32ea597305 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:32.837678 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf8e78-0fae-4277-82dd-1c32ea597305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:32.853797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b0bf8e78-0fae-4277-82dd-1c32ea597305 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:32.853797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:32.853797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:32.853797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:32.853797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:32.853797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:32.853797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:32.853797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:32.853797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:32.853797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:32.853797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:32.853797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:32.853797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:32.853797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:32.853797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:32.853797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:32.854697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:32.854697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:32.854697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:32.854697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:32.854697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:32.854697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:32.854697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:32.854697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:32.854697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:32.854697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:32.854697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:32.854697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:32.854697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:32.854697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:32.854697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:32.854697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:32.855503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:32.855503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:32.855503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:32.855503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:32.855503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:32.855503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:32.855503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:32.855503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:32.855503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:32.855503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:32.855503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:32.855503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:32.858180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 263/543] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:49:32.864273 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf8e78-0fae-4277-82dd-1c32ea597305 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:32.909981 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246f19d-8090-4c2b-a9e4-f7599d103fee None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:32.911862 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:32.987504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf8e78-0fae-4277-82dd-1c32ea597305 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:32.998618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6a4be-7053-4abe-bed9-b2ce041656ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:33.000016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6a4be-7053-4abe-bed9-b2ce041656ee None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:33.000533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6a4be-7053-4abe-bed9-b2ce041656ee None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:33.001005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6a4be-7053-4abe-bed9-b2ce041656ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:33.002113 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6a4be-7053-4abe-bed9-b2ce041656ee None admin] RBAC: Adding query filter params (role.id=24846e82841b4b05b4d9e9d193a98868, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=74b8c75b9cce442b8c430fab6a3fea17) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:33.007605 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6a4be-7053-4abe-bed9-b2ce041656ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:33.009118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6a4be-7053-4abe-bed9-b2ce041656ee None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:33.025503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 281/544] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:49:32 2022] GET /identity/v3/role_assignments?user.id=74b8c75b9cce442b8c430fab6a3fea17&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=24846e82841b4b05b4d9e9d193a98868 => generated 259 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:33.033172 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc1079-484c-4db2-a146-68b1d9bbbde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:33.034451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc1079-484c-4db2-a146-68b1d9bbbde8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:33.035534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc1079-484c-4db2-a146-68b1d9bbbde8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:33.036020 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc1079-484c-4db2-a146-68b1d9bbbde8 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:33.036988 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc1079-484c-4db2-a146-68b1d9bbbde8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:33.037518 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc1079-484c-4db2-a146-68b1d9bbbde8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:33.039104 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc1079-484c-4db2-a146-68b1d9bbbde8 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:33.050182 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 264/545] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:49:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:33.055897 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6a4be-7053-4abe-bed9-b2ce041656ee None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:33.189081 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6a4be-7053-4abe-bed9-b2ce041656ee None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:33.226853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a35b2-0092-40c1-9032-2202c66e63a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:33.227704 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a35b2-0092-40c1-9032-2202c66e63a0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:33.227991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a35b2-0092-40c1-9032-2202c66e63a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:33.228270 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a35b2-0092-40c1-9032-2202c66e63a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:33.239214 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a35b2-0092-40c1-9032-2202c66e63a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:33.240639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a35b2-0092-40c1-9032-2202c66e63a0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:33.258472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-712a35b2-0092-40c1-9032-2202c66e63a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:33.258472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:33.258472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:33.258472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:33.258472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:33.258472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:33.258472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:33.258472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:33.258472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:33.258472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:33.258472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:33.258472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:33.258472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:33.258472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:33.258472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:33.259872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:33.259872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:33.259872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:33.259872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:33.259872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:33.259872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:33.259872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:33.259872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:33.259872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:33.259872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:33.259872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:33.259872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:33.259872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:33.259872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:33.259872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:33.259872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:33.259872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:33.260717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:33.260717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:33.260717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:33.260717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:33.260717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 282/546] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:33.265378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc1079-484c-4db2-a146-68b1d9bbbde8 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:33.292274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:33.406967 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc1079-484c-4db2-a146-68b1d9bbbde8 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:33.451972 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc6318-be72-4207-b3a7-10d927b5acef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:33.452936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc6318-be72-4207-b3a7-10d927b5acef None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:33.453393 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc6318-be72-4207-b3a7-10d927b5acef None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:33.453873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc6318-be72-4207-b3a7-10d927b5acef None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:33.454768 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc6318-be72-4207-b3a7-10d927b5acef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:33.455270 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc6318-be72-4207-b3a7-10d927b5acef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:33.457387 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc6318-be72-4207-b3a7-10d927b5acef None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:33.476155 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 265/547] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:33.481843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a35b2-0092-40c1-9032-2202c66e63a0 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:33.512709 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:33.611062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a35b2-0092-40c1-9032-2202c66e63a0 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:33.646592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d467dd-3a28-4256-b4d1-90ad1b4d3810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:33.647423 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d467dd-3a28-4256-b4d1-90ad1b4d3810 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:33.647718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d467dd-3a28-4256-b4d1-90ad1b4d3810 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:33.648233 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d467dd-3a28-4256-b4d1-90ad1b4d3810 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:33.655972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d467dd-3a28-4256-b4d1-90ad1b4d3810 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:33.657432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d467dd-3a28-4256-b4d1-90ad1b4d3810 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:33.672278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-02d467dd-3a28-4256-b4d1-90ad1b4d3810 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 18:49:33.672278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:33.672278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:33.672278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:33.672278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:33.672278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:33.672278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:33.672278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:33.672278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:33.672278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:33.672278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:33.672278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:33.672278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:33.672278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:33.672278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:33.672278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:33.673358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:33.673358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:33.673358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:33.673358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:33.673358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:33.673358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:33.673358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:33.673358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:33.673358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:33.673358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:33.673358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:33.673358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:33.673358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:33.673358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:33.673358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:33.673358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:33.673358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:33.673358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:33.674250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:33.674250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:33.674250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:33.674250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:33.674250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:33.674250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:33.674250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:33.674250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:33.674250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 18:49:33.674250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:33.674250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 283/548] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:49:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:33.680239 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc6318-be72-4207-b3a7-10d927b5acef None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:33.804382 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:33.807449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabeefa0-b8fa-451d-97bf-ca0894573ab8 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:33.807828 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabeefa0-b8fa-451d-97bf-ca0894573ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:33.808114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabeefa0-b8fa-451d-97bf-ca0894573ab8 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:33.809679 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 284/549] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:49:33.820299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ef6c9-4d0c-4038-adec-2b7cf2f51068 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:33.820591 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ef6c9-4d0c-4038-adec-2b7cf2f51068 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:33.820866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ef6c9-4d0c-4038-adec-2b7cf2f51068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:33.833310 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc6318-be72-4207-b3a7-10d927b5acef None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:33.855566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-417ef6c9-4d0c-4038-adec-2b7cf2f51068 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:33.857257 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-417ef6c9-4d0c-4038-adec-2b7cf2f51068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:33.897610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6427c6-34df-435b-99d8-9c0c3851718b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:33.899558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6427c6-34df-435b-99d8-9c0c3851718b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:33.901024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6427c6-34df-435b-99d8-9c0c3851718b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:33.901779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6427c6-34df-435b-99d8-9c0c3851718b None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:33.902972 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6427c6-34df-435b-99d8-9c0c3851718b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:33.904633 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6427c6-34df-435b-99d8-9c0c3851718b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:33.906978 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6427c6-34df-435b-99d8-9c0c3851718b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:33.940145 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 266/550] 10.222.0.69 () {56 vars in 1135 bytes} [Thu Jul 14 18:49:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:33.942160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4b14c-7fb0-4482-996a-1d783e716a07 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:33.942455 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4b14c-7fb0-4482-996a-1d783e716a07 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:33.942738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4b14c-7fb0-4482-996a-1d783e716a07 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:33.948388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 267/551] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:33.949190 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4b14c-7fb0-4482-996a-1d783e716a07 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:33.971678 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 285/552] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:33.973339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ece0b3-02ba-41f6-97b2-537f3755d0d2 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:33.973916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ece0b3-02ba-41f6-97b2-537f3755d0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:33.974404 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ece0b3-02ba-41f6-97b2-537f3755d0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:33.992151 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d9ece0b3-02ba-41f6-97b2-537f3755d0d2 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:33.993861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d9ece0b3-02ba-41f6-97b2-537f3755d0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:34.002927 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:34.076088 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 286/553] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:34.077262 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ece0b3-02ba-41f6-97b2-537f3755d0d2 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:34.103853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4b14c-7fb0-4482-996a-1d783e716a07 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:34.143508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:34.150099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0437fd5-d5dc-49fd-bca0-e34d7434b757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:34.150757 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0437fd5-d5dc-49fd-bca0-e34d7434b757 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:34.151093 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0437fd5-d5dc-49fd-bca0-e34d7434b757 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:34.151767 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0437fd5-d5dc-49fd-bca0-e34d7434b757 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:34.210685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ece0b3-02ba-41f6-97b2-537f3755d0d2 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:34.242060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0437fd5-d5dc-49fd-bca0-e34d7434b757 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:34.243868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0437fd5-d5dc-49fd-bca0-e34d7434b757 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:34.253747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620bc69-c9c5-490a-9f14-eb997912ed0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:34.255255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620bc69-c9c5-490a-9f14-eb997912ed0d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:34.255745 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620bc69-c9c5-490a-9f14-eb997912ed0d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:34.256216 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620bc69-c9c5-490a-9f14-eb997912ed0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:34.265570 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:34.271184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620bc69-c9c5-490a-9f14-eb997912ed0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:34.283527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c620bc69-c9c5-490a-9f14-eb997912ed0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:34.283527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:34.283527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:34.283527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:34.283527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:34.283527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:34.283527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:34.283527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:34.283527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:34.283527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:34.283527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:34.283527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:34.283527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:34.283527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:34.283527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:34.283527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:34.284957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:34.284957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:34.284957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:34.284957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:34.284957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:34.284957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:34.284957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:34.284957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:34.284957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:34.284957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:34.284957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:34.284957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:34.284957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:34.284957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:34.284957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:34.284957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:34.285890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:34.285890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:34.285890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:34.285890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:34.285890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:34.285890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:34.285890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:34.285890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:34.285890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:34.285890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:34.285890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:34.285890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:34.285890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 287/554] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:34.291059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620bc69-c9c5-490a-9f14-eb997912ed0d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:34.364041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 268/555] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:34.365500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0437fd5-d5dc-49fd-bca0-e34d7434b757 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:34.396473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:34.410836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620bc69-c9c5-490a-9f14-eb997912ed0d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:34.446510 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe4868-5a53-4eb4-a1a0-cdde2edca23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:34.447531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe4868-5a53-4eb4-a1a0-cdde2edca23b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:34.447819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe4868-5a53-4eb4-a1a0-cdde2edca23b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:34.448131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe4868-5a53-4eb4-a1a0-cdde2edca23b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:34.461367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe4868-5a53-4eb4-a1a0-cdde2edca23b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:34.469813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-1afe4868-5a53-4eb4-a1a0-cdde2edca23b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:34.469813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:34.469813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:34.469813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:34.469813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:34.469813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:34.469813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:34.469813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:34.469813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:34.469813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:34.469813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:34.469813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:34.469813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:34.469813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:34.469813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:34.469813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:34.470708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:34.470708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:34.470708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:34.470708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:34.470708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:34.470708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:34.470708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:34.470708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:34.470708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:34.470708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:34.470708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:34.470708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:34.470708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:34.470708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:34.470708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:34.470708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:34.471616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:34.471616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:34.471616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:34.471616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:34.471616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:34.471616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:34.471616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:34.471616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:34.471616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:34.471616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:34.471616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:34.471616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:34.473392 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0437fd5-d5dc-49fd-bca0-e34d7434b757 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:34.476647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 288/556] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:34.477700 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe4868-5a53-4eb4-a1a0-cdde2edca23b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:34.509461 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05595fdf-7b8d-4634-89a3-c9b52d08ec9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:34.510505 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05595fdf-7b8d-4634-89a3-c9b52d08ec9f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:34.511020 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05595fdf-7b8d-4634-89a3-c9b52d08ec9f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:34.511695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05595fdf-7b8d-4634-89a3-c9b52d08ec9f None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:34.512632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05595fdf-7b8d-4634-89a3-c9b52d08ec9f None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:34.513314 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05595fdf-7b8d-4634-89a3-c9b52d08ec9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:34.514881 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05595fdf-7b8d-4634-89a3-c9b52d08ec9f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:34.524323 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 269/557] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:49:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:34.525610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05595fdf-7b8d-4634-89a3-c9b52d08ec9f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:34.546356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:34.589537 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe4868-5a53-4eb4-a1a0-cdde2edca23b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:34.623620 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:34.625936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b967c-bad2-439a-8ba1-6c2e5ddec679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:34.626744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b967c-bad2-439a-8ba1-6c2e5ddec679 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:34.627087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b967c-bad2-439a-8ba1-6c2e5ddec679 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:34.627396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b967c-bad2-439a-8ba1-6c2e5ddec679 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:34.631722 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05595fdf-7b8d-4634-89a3-c9b52d08ec9f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:34.639090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b967c-bad2-439a-8ba1-6c2e5ddec679 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:34.640609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b967c-bad2-439a-8ba1-6c2e5ddec679 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:34.660015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-204b967c-bad2-439a-8ba1-6c2e5ddec679 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:34.660015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:34.660015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:34.660015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:34.660015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:34.660015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:34.660015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:34.660015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:34.660015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:34.660015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:34.660015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:34.660015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:34.660015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:34.660015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:34.660015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:34.661424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:34.661424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:34.661424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:34.661424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:34.661424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:34.661424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:34.661424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:34.661424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:34.661424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:34.661424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:34.661424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:34.661424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:34.661424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:34.661424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:34.661424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:34.661424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:34.661424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:34.662273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:34.662273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:34.662273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:34.662273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:34.665967 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 289/558] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:34.667058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b967c-bad2-439a-8ba1-6c2e5ddec679 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:34.669467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf754123-d172-4423-9e2e-5447d562d30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:34.670269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf754123-d172-4423-9e2e-5447d562d30a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:34.670646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf754123-d172-4423-9e2e-5447d562d30a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:34.670995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf754123-d172-4423-9e2e-5447d562d30a None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:34.671825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf754123-d172-4423-9e2e-5447d562d30a None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:34.672215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf754123-d172-4423-9e2e-5447d562d30a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:34.673642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf754123-d172-4423-9e2e-5447d562d30a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:34.682489 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 270/559] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:49:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:34.683432 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf754123-d172-4423-9e2e-5447d562d30a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:34.785642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b967c-bad2-439a-8ba1-6c2e5ddec679 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:34.796189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:34.797565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf754123-d172-4423-9e2e-5447d562d30a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:34.820153 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b892fb-e98c-4a7a-9c40-82c7622e5bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:34.822925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b892fb-e98c-4a7a-9c40-82c7622e5bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:34.822925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b892fb-e98c-4a7a-9c40-82c7622e5bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:34.822925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b892fb-e98c-4a7a-9c40-82c7622e5bb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:34.832768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b892fb-e98c-4a7a-9c40-82c7622e5bb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:34.834256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b892fb-e98c-4a7a-9c40-82c7622e5bb3 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:34.835220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fb94a-06d5-43b1-98f1-88556f049246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:34.835933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fb94a-06d5-43b1-98f1-88556f049246 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:34.836217 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fb94a-06d5-43b1-98f1-88556f049246 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:34.836497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fb94a-06d5-43b1-98f1-88556f049246 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:34.837229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3fb94a-06d5-43b1-98f1-88556f049246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:34.837544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3fb94a-06d5-43b1-98f1-88556f049246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:34.838904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3fb94a-06d5-43b1-98f1-88556f049246 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:34.851048 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 271/560] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:34.851890 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fb94a-06d5-43b1-98f1-88556f049246 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:34.853934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-32b892fb-e98c-4a7a-9c40-82c7622e5bb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:34.853934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:34.853934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:34.853934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:34.853934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:34.853934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:34.853934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:34.853934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:34.853934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:34.853934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:34.853934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:34.853934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:34.853934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:34.853934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:34.853934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:34.854817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:34.854817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:34.854817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:34.854817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:34.854817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:34.854817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:34.854817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:34.854817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:34.854817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:34.854817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:34.854817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:34.854817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:34.854817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:34.854817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:34.854817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:34.854817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:34.854817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:34.856252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:34.856252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:34.856252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:34.856252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:34.859499 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 290/561] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:34.860853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7280e44-d30f-4b44-9ee1-d9d4978eabfc None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:34.861144 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7280e44-d30f-4b44-9ee1-d9d4978eabfc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:34.861425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7280e44-d30f-4b44-9ee1-d9d4978eabfc None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:34.866909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 291/562] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:34.866909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7280e44-d30f-4b44-9ee1-d9d4978eabfc None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:34.877253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:34.968746 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fb94a-06d5-43b1-98f1-88556f049246 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:34.972495 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7280e44-d30f-4b44-9ee1-d9d4978eabfc None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:35.012153 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03435835-c7e5-4267-b455-3383a16a79d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:35.013114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03435835-c7e5-4267-b455-3383a16a79d7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:35.013415 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03435835-c7e5-4267-b455-3383a16a79d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:35.013673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03435835-c7e5-4267-b455-3383a16a79d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:35.015048 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47c4da-3249-423b-857e-6502cd39ac50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:35.015968 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47c4da-3249-423b-857e-6502cd39ac50 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:35.016348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47c4da-3249-423b-857e-6502cd39ac50 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:35.016733 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47c4da-3249-423b-857e-6502cd39ac50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:35.024497 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47c4da-3249-423b-857e-6502cd39ac50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:35.024964 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03435835-c7e5-4267-b455-3383a16a79d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:35.026183 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47c4da-3249-423b-857e-6502cd39ac50 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:35.026676 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03435835-c7e5-4267-b455-3383a16a79d7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:35.040849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-1a47c4da-3249-423b-857e-6502cd39ac50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:35.040849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:35.040849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:35.040849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:35.040849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:35.040849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:35.040849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:35.040849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:35.040849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:35.040849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:35.040849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:35.040849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:35.040849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:35.040849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:35.040849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:35.041943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:35.041943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:35.041943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:35.041943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:35.041943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:35.041943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:35.041943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:35.041943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:35.041943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:35.041943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:35.041943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:35.041943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:35.041943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:35.041943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:35.041943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:35.041943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:35.041943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:35.042998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:35.042998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:35.042998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:35.042998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:35.042998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:35.042998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:35.042998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:35.042998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:35.049265 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 292/563] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:35.049397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-03435835-c7e5-4267-b455-3383a16a79d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:35.049397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:35.049397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:35.049397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:35.049397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:35.049397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:35.049397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:35.049397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:35.049397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:35.049397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:35.049397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:35.049397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:35.049397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:35.049397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:35.049397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:35.050420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:35.050420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:35.050420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:35.050420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:35.050420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:35.050420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:35.050420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:35.050420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:35.050420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:35.050420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:35.050420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:35.050420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:35.050420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:35.050420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:35.050420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:35.050420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:35.050420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:35.051637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:35.051637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:35.051637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:35.051637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:35.051894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47c4da-3249-423b-857e-6502cd39ac50 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:35.051975 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 272/564] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:35.051975 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384f1af-2848-4003-a196-9fcf69e411a8 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:35.051975 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384f1af-2848-4003-a196-9fcf69e411a8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:35.051975 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384f1af-2848-4003-a196-9fcf69e411a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:35.067390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c384f1af-2848-4003-a196-9fcf69e411a8 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:35.068621 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c384f1af-2848-4003-a196-9fcf69e411a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:35.076497 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:35.135511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:35.151195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 273/565] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:35.151195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384f1af-2848-4003-a196-9fcf69e411a8 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:35.166453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47c4da-3249-423b-857e-6502cd39ac50 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:35.205278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2b148-5cad-4fa9-9930-0de7fdca4c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:35.206238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2b148-5cad-4fa9-9930-0de7fdca4c07 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:35.206518 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2b148-5cad-4fa9-9930-0de7fdca4c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:35.206807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2b148-5cad-4fa9-9930-0de7fdca4c07 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:35.207560 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2b148-5cad-4fa9-9930-0de7fdca4c07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:35.207871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2b148-5cad-4fa9-9930-0de7fdca4c07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:35.209681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2b148-5cad-4fa9-9930-0de7fdca4c07 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:35.225812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 293/566] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:35.225812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2b148-5cad-4fa9-9930-0de7fdca4c07 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:35.257990 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384f1af-2848-4003-a196-9fcf69e411a8 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:35.292412 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d1924-5d97-4958-a4ed-a635cc96a6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:35.292412 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d1924-5d97-4958-a4ed-a635cc96a6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:35.292706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d1924-5d97-4958-a4ed-a635cc96a6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:35.292958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d1924-5d97-4958-a4ed-a635cc96a6a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:35.293742 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d1924-5d97-4958-a4ed-a635cc96a6a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:35.294053 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d1924-5d97-4958-a4ed-a635cc96a6a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:35.295506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d1924-5d97-4958-a4ed-a635cc96a6a1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:35.307426 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 274/567] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:35.307905 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:35.308344 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d1924-5d97-4958-a4ed-a635cc96a6a1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:35.330219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2b148-5cad-4fa9-9930-0de7fdca4c07 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:35.362429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69373738-051f-4ff2-830d-4d90021704de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:35.363647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69373738-051f-4ff2-830d-4d90021704de None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:35.363932 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69373738-051f-4ff2-830d-4d90021704de None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:35.364209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69373738-051f-4ff2-830d-4d90021704de None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:35.364921 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69373738-051f-4ff2-830d-4d90021704de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:35.365227 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69373738-051f-4ff2-830d-4d90021704de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:35.366599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69373738-051f-4ff2-830d-4d90021704de None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:35.366863 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:35.377610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 294/568] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:35.378323 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69373738-051f-4ff2-830d-4d90021704de None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:35.412284 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d1924-5d97-4958-a4ed-a635cc96a6a1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:35.447946 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94942e9-116e-41dd-a3dd-848d4122cd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:35.448735 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94942e9-116e-41dd-a3dd-848d4122cd58 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:35.449025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94942e9-116e-41dd-a3dd-848d4122cd58 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:35.449303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94942e9-116e-41dd-a3dd-848d4122cd58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:35.459574 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94942e9-116e-41dd-a3dd-848d4122cd58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:35.467313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b94942e9-116e-41dd-a3dd-848d4122cd58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:35.467313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:35.467313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:35.467313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:35.467313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:35.467313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:35.467313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:35.467313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:35.467313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:35.467313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:35.467313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:35.467313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:35.467313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:35.467313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:35.467313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:35.467313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:35.468192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:35.468192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:35.468192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:35.468192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:35.468192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:35.468192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:35.468192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:35.468192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:35.468192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:35.468192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:35.468192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:35.468192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:35.468192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:35.468192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:35.468192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:35.468192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:35.469018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:35.469018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:35.469018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:35.469018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:35.469018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:35.469018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:35.469018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:35.469018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:35.469018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:35.469018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:35.469018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:35.469018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:35.469018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 275/569] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:35.469868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baaed2e-63ac-41bd-83fd-f3889138e13f None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:35.470141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baaed2e-63ac-41bd-83fd-f3889138e13f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:35.470423 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baaed2e-63ac-41bd-83fd-f3889138e13f None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:35.471732 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 276/570] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:35.472450 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baaed2e-63ac-41bd-83fd-f3889138e13f None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:35.479593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69373738-051f-4ff2-830d-4d90021704de None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:35.514007 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19548ae0-d5b4-46f1-b40c-9ca72297172a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:35.514805 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19548ae0-d5b4-46f1-b40c-9ca72297172a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:35.515093 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19548ae0-d5b4-46f1-b40c-9ca72297172a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:35.515402 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19548ae0-d5b4-46f1-b40c-9ca72297172a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:35.522956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19548ae0-d5b4-46f1-b40c-9ca72297172a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:35.524431 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19548ae0-d5b4-46f1-b40c-9ca72297172a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:35.538991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-19548ae0-d5b4-46f1-b40c-9ca72297172a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 18:49:35.538991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:35.538991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:35.538991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:35.538991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:35.538991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:35.538991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:35.538991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:35.538991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:35.538991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:35.538991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:35.538991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:35.538991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:35.538991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:35.538991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:35.538991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:35.540051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:35.540051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:35.540051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:35.540051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:35.540051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:35.540051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:35.540051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:35.540051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:35.540051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:35.540051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:35.540051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:35.540051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:35.540051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:35.540051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:35.540051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:35.540051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:35.540051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:35.540051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:35.540977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:35.540977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:35.540977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:35.540977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:35.540977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:35.540977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:35.540977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:35.540977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:35.540977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 18:49:35.540977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:35.540977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 295/571] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:35.540977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19548ae0-d5b4-46f1-b40c-9ca72297172a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:35.543762 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:35.579607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baaed2e-63ac-41bd-83fd-f3889138e13f None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:35.596567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:35.614026 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912be3c-b1b9-4d8d-b197-5d3ba727f139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:35.614685 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912be3c-b1b9-4d8d-b197-5d3ba727f139 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:35.614986 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912be3c-b1b9-4d8d-b197-5d3ba727f139 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:35.615315 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912be3c-b1b9-4d8d-b197-5d3ba727f139 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:35.616145 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912be3c-b1b9-4d8d-b197-5d3ba727f139 None admin] RBAC: Adding query filter params (role.id=24846e82841b4b05b4d9e9d193a98868, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=4368f68724f946eca53084a9ec7ff590) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:35.616457 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912be3c-b1b9-4d8d-b197-5d3ba727f139 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:35.617889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912be3c-b1b9-4d8d-b197-5d3ba727f139 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:35.623260 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 277/572] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:49:35 2022] GET /identity/v3/role_assignments?user.id=4368f68724f946eca53084a9ec7ff590&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=24846e82841b4b05b4d9e9d193a98868 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:35.624034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912be3c-b1b9-4d8d-b197-5d3ba727f139 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:35.639824 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19548ae0-d5b4-46f1-b40c-9ca72297172a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:35.678454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033a6b5-602c-4cfc-abfa-b13233a27fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:35.679239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033a6b5-602c-4cfc-abfa-b13233a27fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:35.679528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033a6b5-602c-4cfc-abfa-b13233a27fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:35.679801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033a6b5-602c-4cfc-abfa-b13233a27fc0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:35.688503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033a6b5-602c-4cfc-abfa-b13233a27fc0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:35.689992 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033a6b5-602c-4cfc-abfa-b13233a27fc0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:35.705414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-0033a6b5-602c-4cfc-abfa-b13233a27fc0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 18:49:35.705414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:35.705414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:35.705414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:35.705414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:35.705414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:35.705414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:35.705414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:35.705414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:35.705414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:35.705414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:35.705414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:35.705414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:35.705414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:35.705414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:35.705414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:35.706619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:35.706619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:35.706619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:35.706619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:35.706619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:35.706619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:35.706619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:35.706619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:35.706619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:35.706619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:35.706619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:35.706619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:35.706619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:35.706619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:35.706619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:35.706619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:35.706619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:35.706619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:35.707607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:35.707607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:35.707607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:35.707607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:35.707607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:35.707607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:35.707607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:35.707607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:35.707607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 18:49:35.707607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:35.707607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 296/573] 10.222.0.69 () {56 vars in 1093 bytes} [Thu Jul 14 18:49:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:35.708184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0f8a1-a1ac-4386-befd-13c15f092c7b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:35.708273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0f8a1-a1ac-4386-befd-13c15f092c7b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:35.708704 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0f8a1-a1ac-4386-befd-13c15f092c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:35.728193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-a8f0f8a1-a1ac-4386-befd-13c15f092c7b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:35.729409 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a8f0f8a1-a1ac-4386-befd-13c15f092c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:35.766030 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912be3c-b1b9-4d8d-b197-5d3ba727f139 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:35.794040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:35.804691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 297/574] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:35.805107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca780f-6814-4a8a-80dd-3450e10917be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:35.805788 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca780f-6814-4a8a-80dd-3450e10917be None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:35.806084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca780f-6814-4a8a-80dd-3450e10917be None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:35.806366 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca780f-6814-4a8a-80dd-3450e10917be None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:35.806851 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0f8a1-a1ac-4386-befd-13c15f092c7b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:35.807122 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca780f-6814-4a8a-80dd-3450e10917be None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:35.807469 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca780f-6814-4a8a-80dd-3450e10917be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:35.808988 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca780f-6814-4a8a-80dd-3450e10917be None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:35.829071 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 278/575] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:49:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:35.829976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca780f-6814-4a8a-80dd-3450e10917be None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:35.877593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:35.916869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0f8a1-a1ac-4386-befd-13c15f092c7b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:35.935852 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca780f-6814-4a8a-80dd-3450e10917be None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:35.952807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f6b05-1c5d-43c5-b2b2-583c6097f261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:35.953489 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f6b05-1c5d-43c5-b2b2-583c6097f261 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:35.953779 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f6b05-1c5d-43c5-b2b2-583c6097f261 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:35.954065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f6b05-1c5d-43c5-b2b2-583c6097f261 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:35.954889 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f6b05-1c5d-43c5-b2b2-583c6097f261 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:35.955225 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f6b05-1c5d-43c5-b2b2-583c6097f261 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:35.957205 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f6b05-1c5d-43c5-b2b2-583c6097f261 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:35.973086 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc44fe-9f72-4ce4-807b-086f794ab46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:35.973748 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc44fe-9f72-4ce4-807b-086f794ab46d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:35.974034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc44fe-9f72-4ce4-807b-086f794ab46d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:35.974307 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc44fe-9f72-4ce4-807b-086f794ab46d None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:35.975131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc44fe-9f72-4ce4-807b-086f794ab46d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:35.975482 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc44fe-9f72-4ce4-807b-086f794ab46d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:35.980234 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 298/576] 10.222.0.69 () {56 vars in 1137 bytes} [Thu Jul 14 18:49:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:35.980234 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f6b05-1c5d-43c5-b2b2-583c6097f261 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:35.980474 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc44fe-9f72-4ce4-807b-086f794ab46d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:36.003665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 279/577] 10.222.0.69 () {56 vars in 1133 bytes} [Thu Jul 14 18:49:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:36.004593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc44fe-9f72-4ce4-807b-086f794ab46d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:36.033789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:36.086263 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f6b05-1c5d-43c5-b2b2-583c6097f261 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:36.111544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc44fe-9f72-4ce4-807b-086f794ab46d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:36.113297 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:36.123762 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784b3c7-2c23-4061-b473-cbdc3f2ba494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:36.124568 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784b3c7-2c23-4061-b473-cbdc3f2ba494 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:36.127334 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784b3c7-2c23-4061-b473-cbdc3f2ba494 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:36.127334 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784b3c7-2c23-4061-b473-cbdc3f2ba494 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:36.136217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784b3c7-2c23-4061-b473-cbdc3f2ba494 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:36.144557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-b784b3c7-2c23-4061-b473-cbdc3f2ba494 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:36.144557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:36.144557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:36.144557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:36.144557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:36.144557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:36.144557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:36.144557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:36.144557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:36.144557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:36.144557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:36.144557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:36.144557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:36.144557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:36.144557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:36.144557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:36.145648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:36.145648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:36.145648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:36.145648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:36.145648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:36.145648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:36.145648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:36.145648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:36.145648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:36.145648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:36.145648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:36.145648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:36.145648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:36.145648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:36.145648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:36.145648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:36.151807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:36.151807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:36.151807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:36.151807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:36.151807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:36.151807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:36.151807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:36.151807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:36.151807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:36.151807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:36.151807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:36.151807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:36.151807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 299/578] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:36.151807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784b3c7-2c23-4061-b473-cbdc3f2ba494 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:36.156098 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc9b1f-ea6c-4ec6-83bb-1210fb316dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:36.156581 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc9b1f-ea6c-4ec6-83bb-1210fb316dee None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:36.156939 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc9b1f-ea6c-4ec6-83bb-1210fb316dee None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:36.157290 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc9b1f-ea6c-4ec6-83bb-1210fb316dee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:36.170207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc9b1f-ea6c-4ec6-83bb-1210fb316dee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:36.171771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc9b1f-ea6c-4ec6-83bb-1210fb316dee None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:36.189583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-accc9b1f-ea6c-4ec6-83bb-1210fb316dee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:36.189583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:36.189583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:36.189583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:36.189583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:36.189583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:36.189583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:36.189583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:36.189583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:36.189583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:36.189583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:36.189583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:36.189583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:36.189583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:36.189583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:36.190646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:36.190646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:36.190646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:36.190646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:36.190646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:36.190646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:36.190646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:36.190646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:36.190646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:36.190646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:36.190646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:36.190646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:36.190646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:36.190646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:36.190646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:36.190646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:36.190646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:36.191878 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:36.191878 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:36.191878 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:36.191878 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:36.196720 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 280/579] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:36.197361 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc9b1f-ea6c-4ec6-83bb-1210fb316dee None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:36.253312 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784b3c7-2c23-4061-b473-cbdc3f2ba494 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:36.264012 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:36.291158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28172c2-b430-4628-a9c8-4e3c82db8166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:36.291992 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28172c2-b430-4628-a9c8-4e3c82db8166 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:36.292282 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28172c2-b430-4628-a9c8-4e3c82db8166 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:36.292627 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28172c2-b430-4628-a9c8-4e3c82db8166 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:36.301366 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc9b1f-ea6c-4ec6-83bb-1210fb316dee None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:36.303840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28172c2-b430-4628-a9c8-4e3c82db8166 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:36.305435 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28172c2-b430-4628-a9c8-4e3c82db8166 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:36.324240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-e28172c2-b430-4628-a9c8-4e3c82db8166 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:36.324240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:36.324240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:36.324240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:36.324240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:36.324240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:36.324240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:36.324240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:36.324240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:36.324240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:36.324240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:36.324240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:36.324240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:36.324240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:36.324240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:36.325747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:36.325747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:36.325747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:36.325747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:36.325747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:36.325747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:36.325747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:36.325747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:36.325747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:36.325747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:36.325747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:36.325747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:36.325747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:36.325747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:36.325747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:36.325747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:36.325747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:36.326818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:36.326818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:36.326818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:36.326818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:36.331836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 300/580] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:36.332876 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28172c2-b430-4628-a9c8-4e3c82db8166 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:36.339917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a1623-c110-4f7d-931e-0263a41542e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:36.340720 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a1623-c110-4f7d-931e-0263a41542e9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:36.341009 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a1623-c110-4f7d-931e-0263a41542e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:36.341290 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a1623-c110-4f7d-931e-0263a41542e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:36.351730 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a1623-c110-4f7d-931e-0263a41542e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:36.353189 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a1623-c110-4f7d-931e-0263a41542e9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:36.374517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b24a1623-c110-4f7d-931e-0263a41542e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:36.374517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:36.374517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:36.374517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:36.374517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:36.374517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:36.374517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:36.374517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:36.374517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:36.374517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:36.374517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:36.374517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:36.374517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:36.374517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:36.374517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:36.376178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:36.376178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:36.376178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:36.376178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:36.376178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:36.376178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:36.376178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:36.376178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:36.376178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:36.376178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:36.376178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:36.376178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:36.376178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:36.376178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:36.376178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:36.376178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:36.376178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:36.382615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:36.382615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:36.382615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:36.382615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:36.382615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 281/581] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:36.382615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a1623-c110-4f7d-931e-0263a41542e9 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:36.389476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:36.437190 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28172c2-b430-4628-a9c8-4e3c82db8166 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:36.474549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d50c3-ce6c-44f0-bda2-3cd6b53f4127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:36.475139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d50c3-ce6c-44f0-bda2-3cd6b53f4127 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:36.475460 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d50c3-ce6c-44f0-bda2-3cd6b53f4127 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:36.475741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d50c3-ce6c-44f0-bda2-3cd6b53f4127 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:36.476589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d50c3-ce6c-44f0-bda2-3cd6b53f4127 None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:36.476794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d50c3-ce6c-44f0-bda2-3cd6b53f4127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:36.478171 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d50c3-ce6c-44f0-bda2-3cd6b53f4127 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:36.484636 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a1623-c110-4f7d-931e-0263a41542e9 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:36.486085 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 301/582] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:49:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:36.486872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d50c3-ce6c-44f0-bda2-3cd6b53f4127 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:36.523281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:36.527380 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f766c-b2da-4499-a754-d4469b01699e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:36.528075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f766c-b2da-4499-a754-d4469b01699e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:36.528432 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f766c-b2da-4499-a754-d4469b01699e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:36.528643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f766c-b2da-4499-a754-d4469b01699e None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:36.529367 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f766c-b2da-4499-a754-d4469b01699e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:36.533862 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f766c-b2da-4499-a754-d4469b01699e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:36.533862 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f766c-b2da-4499-a754-d4469b01699e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:36.547679 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 282/583] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:36.547679 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f766c-b2da-4499-a754-d4469b01699e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:36.612858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d50c3-ce6c-44f0-bda2-3cd6b53f4127 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:36.618344 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:36.656647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66879c-d203-47f6-a493-ba7eb7c3215e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:36.657330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66879c-d203-47f6-a493-ba7eb7c3215e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:36.657621 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66879c-d203-47f6-a493-ba7eb7c3215e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:36.657904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66879c-d203-47f6-a493-ba7eb7c3215e None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:36.658077 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f766c-b2da-4499-a754-d4469b01699e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:36.658637 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66879c-d203-47f6-a493-ba7eb7c3215e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:36.658949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66879c-d203-47f6-a493-ba7eb7c3215e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:36.661447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66879c-d203-47f6-a493-ba7eb7c3215e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:36.675493 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 302/584] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:36.676006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66879c-d203-47f6-a493-ba7eb7c3215e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:36.693053 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34037a-1209-492c-b0ba-9fd45a8c1ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:36.693722 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34037a-1209-492c-b0ba-9fd45a8c1ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:36.694004 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34037a-1209-492c-b0ba-9fd45a8c1ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:36.694280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34037a-1209-492c-b0ba-9fd45a8c1ab5 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:36.694996 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34037a-1209-492c-b0ba-9fd45a8c1ab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:36.695337 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34037a-1209-492c-b0ba-9fd45a8c1ab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:36.696718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34037a-1209-492c-b0ba-9fd45a8c1ab5 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:36.713765 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 283/585] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:36.714623 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34037a-1209-492c-b0ba-9fd45a8c1ab5 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:36.771968 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:36.788619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66879c-d203-47f6-a493-ba7eb7c3215e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:36.815356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34037a-1209-492c-b0ba-9fd45a8c1ab5 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:36.827852 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d79bb3-f07d-48aa-aefb-6a3fa9f975f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:36.828649 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d79bb3-f07d-48aa-aefb-6a3fa9f975f3 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:36.828935 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d79bb3-f07d-48aa-aefb-6a3fa9f975f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:36.829305 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d79bb3-f07d-48aa-aefb-6a3fa9f975f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:36.840758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d79bb3-f07d-48aa-aefb-6a3fa9f975f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:36.842216 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d79bb3-f07d-48aa-aefb-6a3fa9f975f3 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:36.842942 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:36.851467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d474c6-5c95-4973-81d0-bfbe14480c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:36.853414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d474c6-5c95-4973-81d0-bfbe14480c27 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:36.853414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d474c6-5c95-4973-81d0-bfbe14480c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:36.853414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d474c6-5c95-4973-81d0-bfbe14480c27 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:36.860379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d474c6-5c95-4973-81d0-bfbe14480c27 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:36.860501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-82d79bb3-f07d-48aa-aefb-6a3fa9f975f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:36.860501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:36.860501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:36.860501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:36.860501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:36.860501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:36.860501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:36.860501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:36.860501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:36.860501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:36.860501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:36.860501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:36.860501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:36.860501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:36.860501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:36.861313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:36.861313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:36.861313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:36.861313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:36.861313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:36.861313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:36.861313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:36.861313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:36.861313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:36.861313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:36.861313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:36.861313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:36.861313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:36.861313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:36.861313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:36.861313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:36.861313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:36.862172 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:36.862172 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:36.862172 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:36.862172 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:36.862172 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 303/586] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:36.863478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d79bb3-f07d-48aa-aefb-6a3fa9f975f3 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:36.866869 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d474c6-5c95-4973-81d0-bfbe14480c27 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:36.883364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-24d474c6-5c95-4973-81d0-bfbe14480c27 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 18:49:36.883364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:36.883364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:36.883364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:36.883364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:36.883364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:36.883364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:36.883364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:36.883364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:36.883364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:36.883364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:36.883364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:36.883364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:36.883364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:36.883364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:36.883364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:36.884301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:36.884301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:36.884301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:36.884301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:36.884301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:36.884301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:36.884301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:36.884301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:36.884301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:36.884301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:36.884301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:36.884301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:36.884301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:36.884301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:36.884301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:36.884301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:36.884301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:36.884301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:36.885226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:36.885226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:36.885226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:36.885226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:36.885226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:36.885226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:36.885226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:36.885226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:36.885226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 18:49:36.885226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:36.885226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 284/587] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:36.885822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d474c6-5c95-4973-81d0-bfbe14480c27 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:36.982415 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d79bb3-f07d-48aa-aefb-6a3fa9f975f3 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:36.988782 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d474c6-5c95-4973-81d0-bfbe14480c27 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:37.024725 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321e7c1-6c45-4bcc-b51e-89dafed0bcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:37.025446 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321e7c1-6c45-4bcc-b51e-89dafed0bcea None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:37.025733 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321e7c1-6c45-4bcc-b51e-89dafed0bcea None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:37.026014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321e7c1-6c45-4bcc-b51e-89dafed0bcea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:37.026300 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ed4a6-b2a6-41c6-9d97-639b93324ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:37.027398 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ed4a6-b2a6-41c6-9d97-639b93324ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:37.027692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ed4a6-b2a6-41c6-9d97-639b93324ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:37.027974 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ed4a6-b2a6-41c6-9d97-639b93324ba4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:37.029624 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:37.034187 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321e7c1-6c45-4bcc-b51e-89dafed0bcea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:37.035723 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ed4a6-b2a6-41c6-9d97-639b93324ba4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:37.036571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321e7c1-6c45-4bcc-b51e-89dafed0bcea None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:37.037218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ed4a6-b2a6-41c6-9d97-639b93324ba4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:37.051641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-4321e7c1-6c45-4bcc-b51e-89dafed0bcea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:37.051641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:37.051641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:37.051641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:37.051641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:37.051641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:37.051641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:37.051641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:37.051641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:37.051641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:37.051641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:37.051641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:37.051641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:37.051641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:37.051641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:37.052521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:37.052521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:37.052521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:37.052521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:37.052521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:37.052521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:37.052521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:37.052521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:37.052521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:37.052521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:37.052521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:37.052521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:37.052521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:37.052521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:37.052521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:37.052521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:37.052521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:37.058607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:37.058607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:37.058607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:37.058607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:37.058607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:37.058607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:37.058607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:37.058607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:37.059033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-ca2ed4a6-b2a6-41c6-9d97-639b93324ba4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:37.059033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:37.059033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:37.059033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:37.059033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:37.059033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:37.059033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:37.059033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:37.059033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:37.059033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:37.059033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:37.059033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:37.059033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:37.059033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:37.059033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:37.059847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:37.059847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:37.059847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:37.059847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:37.059847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:37.059847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:37.059847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:37.059847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:37.059847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:37.059847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:37.059847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:37.059847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:37.059847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:37.059847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:37.059847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:37.059847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:37.059847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:37.063108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:37.063108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:37.063108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:37.063108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:37.063108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:37.063108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:37.063108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:37.063108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:37.063108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 285/589] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:37.063108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ed4a6-b2a6-41c6-9d97-639b93324ba4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:37.065218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 304/588] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:37.065218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321e7c1-6c45-4bcc-b51e-89dafed0bcea None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:37.106529 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:37.166014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ed4a6-b2a6-41c6-9d97-639b93324ba4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:37.169397 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321e7c1-6c45-4bcc-b51e-89dafed0bcea None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:37.202507 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea63269-6120-4181-80af-509659c89c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:37.203188 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea63269-6120-4181-80af-509659c89c2f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:37.203593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea63269-6120-4181-80af-509659c89c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:37.203879 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea63269-6120-4181-80af-509659c89c2f None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:37.204059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0fc17-b592-4be7-ae79-bee5451d563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:37.204604 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea63269-6120-4181-80af-509659c89c2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:37.204757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0fc17-b592-4be7-ae79-bee5451d563a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:37.204918 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea63269-6120-4181-80af-509659c89c2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:37.205023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0fc17-b592-4be7-ae79-bee5451d563a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:37.205285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0fc17-b592-4be7-ae79-bee5451d563a None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:37.206119 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0fc17-b592-4be7-ae79-bee5451d563a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:37.206320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea63269-6120-4181-80af-509659c89c2f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:37.206487 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0fc17-b592-4be7-ae79-bee5451d563a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:37.207922 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0fc17-b592-4be7-ae79-bee5451d563a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:37.223651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 286/590] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:37.223651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea63269-6120-4181-80af-509659c89c2f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:37.234545 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 305/591] 10.222.0.69 () {56 vars in 1137 bytes} [Thu Jul 14 18:49:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:37.234545 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0fc17-b592-4be7-ae79-bee5451d563a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:37.295203 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:37.341107 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0fc17-b592-4be7-ae79-bee5451d563a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:37.354439 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea63269-6120-4181-80af-509659c89c2f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:37.363406 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:37.378633 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbc204-ebce-491f-839d-000f8beb70b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:37.379387 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbc204-ebce-491f-839d-000f8beb70b9 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:37.379686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbc204-ebce-491f-839d-000f8beb70b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:37.379951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbc204-ebce-491f-839d-000f8beb70b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:37.380746 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbc204-ebce-491f-839d-000f8beb70b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:37.381056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbc204-ebce-491f-839d-000f8beb70b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:37.382499 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbc204-ebce-491f-839d-000f8beb70b9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:37.391127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14216bda-ecd5-4c25-b740-9f8365de1f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:37.391922 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14216bda-ecd5-4c25-b740-9f8365de1f39 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:37.392215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14216bda-ecd5-4c25-b740-9f8365de1f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:37.392497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14216bda-ecd5-4c25-b740-9f8365de1f39 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:37.393291 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14216bda-ecd5-4c25-b740-9f8365de1f39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:37.393685 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14216bda-ecd5-4c25-b740-9f8365de1f39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:37.399586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 306/592] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:37.399720 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14216bda-ecd5-4c25-b740-9f8365de1f39 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:37.399797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbc204-ebce-491f-839d-000f8beb70b9 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:37.408801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 287/593] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:37.410957 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14216bda-ecd5-4c25-b740-9f8365de1f39 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:37.509768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbc204-ebce-491f-839d-000f8beb70b9 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:37.515055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14216bda-ecd5-4c25-b740-9f8365de1f39 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:37.525152 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:37.546528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd506aa-d962-403e-93cc-1a9041be7702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:37.547296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd506aa-d962-403e-93cc-1a9041be7702 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:37.547589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd506aa-d962-403e-93cc-1a9041be7702 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:37.548188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd506aa-d962-403e-93cc-1a9041be7702 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:37.555137 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf318b-3f65-46c3-aaf6-ecfe52b1b569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:37.555982 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf318b-3f65-46c3-aaf6-ecfe52b1b569 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:37.556273 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf318b-3f65-46c3-aaf6-ecfe52b1b569 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:37.556622 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf318b-3f65-46c3-aaf6-ecfe52b1b569 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:37.560876 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd506aa-d962-403e-93cc-1a9041be7702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:37.562372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd506aa-d962-403e-93cc-1a9041be7702 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:37.564944 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf318b-3f65-46c3-aaf6-ecfe52b1b569 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:37.566441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf318b-3f65-46c3-aaf6-ecfe52b1b569 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:37.581114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-29bf318b-3f65-46c3-aaf6-ecfe52b1b569 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 18:49:37.581114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:37.581114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:37.581114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:37.581114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:37.581114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:37.581114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:37.581114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:37.581114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:37.581114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:37.581114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:37.581114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:37.581114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:37.581114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:37.581114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:37.581114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:37.582666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:37.582666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:37.582666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:37.582666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:37.582666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:37.582666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:37.582666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:37.582666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:37.582666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:37.582666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:37.582666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:37.582666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:37.582666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:37.582666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:37.582666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:37.582666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:37.582666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:37.582666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:37.588933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:37.588933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:37.588933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:37.588933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:37.588933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:37.588933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:37.588933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:37.588933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:37.588933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 18:49:37.588933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:37.589459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-4dd506aa-d962-403e-93cc-1a9041be7702 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:37.589459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:37.589459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:37.589459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:37.589459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:37.589459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:37.589459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:37.589459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:37.589459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:37.589459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:37.589459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:37.589459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:37.589459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:37.589459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:37.589459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:37.590241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:37.590241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:37.590241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:37.590241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:37.590241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:37.590241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:37.590241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:37.590241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:37.590241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:37.590241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:37.590241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:37.590241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:37.590241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:37.590241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:37.590241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:37.590241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:37.590241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:37.591156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:37.591156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:37.591156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:37.591156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:37.593808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 288/594] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:37.593808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf318b-3f65-46c3-aaf6-ecfe52b1b569 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:37.594099 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 307/595] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:37.594099 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd506aa-d962-403e-93cc-1a9041be7702 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:37.618650 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:37.692505 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf318b-3f65-46c3-aaf6-ecfe52b1b569 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:37.694626 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd506aa-d962-403e-93cc-1a9041be7702 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:37.726442 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67b489-9356-44ec-8df2-23b6234b9e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:37.727374 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67b489-9356-44ec-8df2-23b6234b9e80 None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:37.727666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67b489-9356-44ec-8df2-23b6234b9e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:37.727947 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67b489-9356-44ec-8df2-23b6234b9e80 None admin] PATH_INFO: `/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/31b09d72314349c6be0f906484895608/roles/24846e82841b4b05b4d9e9d193a98868` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:37.728614 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996f549-fd7e-458f-a2e3-a81511624b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:37.729502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996f549-fd7e-458f-a2e3-a81511624b54 None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:37.729787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996f549-fd7e-458f-a2e3-a81511624b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:37.730062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996f549-fd7e-458f-a2e3-a81511624b54 None admin] PATH_INFO: `/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/8d27d6df5cb64697adebd961dfbec324/roles/24846e82841b4b05b4d9e9d193a98868` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:37.741991 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67b489-9356-44ec-8df2-23b6234b9e80 None admin] RBAC: Authorizing `identity:create_grant(project_id=6003dba8ddd247e7842d28ea7ca5238c, user_id=31b09d72314349c6be0f906484895608, role_id=24846e82841b4b05b4d9e9d193a98868)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:37.743470 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67b489-9356-44ec-8df2-23b6234b9e80 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:37.744248 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996f549-fd7e-458f-a2e3-a81511624b54 None admin] RBAC: Authorizing `identity:create_grant(project_id=6003dba8ddd247e7842d28ea7ca5238c, user_id=8d27d6df5cb64697adebd961dfbec324, role_id=24846e82841b4b05b4d9e9d193a98868)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:37.745681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996f549-fd7e-458f-a2e3-a81511624b54 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:37.838152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 289/596] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:49:37 2022] PUT /identity/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/31b09d72314349c6be0f906484895608/roles/24846e82841b4b05b4d9e9d193a98868 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:49:37.838940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67b489-9356-44ec-8df2-23b6234b9e80 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:37.849646 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 308/597] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:49:37 2022] PUT /identity/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/8d27d6df5cb64697adebd961dfbec324/roles/24846e82841b4b05b4d9e9d193a98868 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:49:37.868176 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996f549-fd7e-458f-a2e3-a81511624b54 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:37.912566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:37.983592 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:37.994377 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67b489-9356-44ec-8df2-23b6234b9e80 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:38.029388 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996f549-fd7e-458f-a2e3-a81511624b54 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:38.042203 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ff6f1-c688-4878-99db-c5a555284231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:38.042891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ff6f1-c688-4878-99db-c5a555284231 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:38.043199 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ff6f1-c688-4878-99db-c5a555284231 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:38.043555 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ff6f1-c688-4878-99db-c5a555284231 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:38.044733 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ff6f1-c688-4878-99db-c5a555284231 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:38.045049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ff6f1-c688-4878-99db-c5a555284231 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:38.046753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ff6f1-c688-4878-99db-c5a555284231 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:38.061461 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 290/598] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:38.062792 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645cf37-73e0-427d-87f7-5f517b623e06 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:38.063089 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645cf37-73e0-427d-87f7-5f517b623e06 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:38.063405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645cf37-73e0-427d-87f7-5f517b623e06 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:38.064771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 291/599] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:38.068305 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645cf37-73e0-427d-87f7-5f517b623e06 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:38.078124 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c38262-03f6-4805-b474-d74531a927b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:38.079101 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c38262-03f6-4805-b474-d74531a927b9 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:38.079618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c38262-03f6-4805-b474-d74531a927b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:38.080090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c38262-03f6-4805-b474-d74531a927b9 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:38.081202 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c38262-03f6-4805-b474-d74531a927b9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:38.082254 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c38262-03f6-4805-b474-d74531a927b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:38.083955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c38262-03f6-4805-b474-d74531a927b9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:38.102435 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 309/600] 10.222.0.69 () {56 vars in 1139 bytes} [Thu Jul 14 18:49:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:38.103672 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979dfb79-28a7-460d-ac5e-74687ccad49c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:38.103969 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979dfb79-28a7-460d-ac5e-74687ccad49c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:38.104249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979dfb79-28a7-460d-ac5e-74687ccad49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:38.121604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-979dfb79-28a7-460d-ac5e-74687ccad49c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:38.122818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-979dfb79-28a7-460d-ac5e-74687ccad49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:38.197487 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645cf37-73e0-427d-87f7-5f517b623e06 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:38.199898 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 310/601] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:38.201013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979dfb79-28a7-460d-ac5e-74687ccad49c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:38.204390 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:38.254819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b23a9-4775-4ae6-848a-ec7a2595ef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:38.255677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b23a9-4775-4ae6-848a-ec7a2595ef57 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:38.256022 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b23a9-4775-4ae6-848a-ec7a2595ef57 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:38.256306 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b23a9-4775-4ae6-848a-ec7a2595ef57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:38.259203 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:38.263952 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b23a9-4775-4ae6-848a-ec7a2595ef57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:38.265668 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b23a9-4775-4ae6-848a-ec7a2595ef57 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:38.284704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-e90b23a9-4775-4ae6-848a-ec7a2595ef57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:38.284704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:38.284704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:38.284704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:38.284704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:38.284704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:38.284704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:38.284704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:38.284704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:38.284704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:38.284704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:38.284704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:38.284704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:38.284704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:38.284704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:38.285653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:38.285653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:38.285653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:38.285653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:38.285653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:38.285653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:38.285653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:38.285653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:38.285653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:38.285653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:38.285653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:38.285653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:38.285653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:38.285653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:38.285653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:38.285653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:38.285653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:38.286610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:38.286610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:38.286610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:38.286610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:38.286610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:38.286610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:38.286610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:38.286610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:38.290552 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 292/602] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:38.291433 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b23a9-4775-4ae6-848a-ec7a2595ef57 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:38.325870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979dfb79-28a7-460d-ac5e-74687ccad49c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:38.373530 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c9fc7-8289-478d-aedf-33b8d8abec93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:38.375114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c9fc7-8289-478d-aedf-33b8d8abec93 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:38.376153 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c9fc7-8289-478d-aedf-33b8d8abec93 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:38.376655 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c9fc7-8289-478d-aedf-33b8d8abec93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:38.391328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c9fc7-8289-478d-aedf-33b8d8abec93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:38.393565 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c9fc7-8289-478d-aedf-33b8d8abec93 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:38.403419 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b23a9-4775-4ae6-848a-ec7a2595ef57 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:38.418478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-3e1c9fc7-8289-478d-aedf-33b8d8abec93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:38.418478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:38.418478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:38.418478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:38.418478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:38.418478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:38.418478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:38.418478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:38.418478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:38.418478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:38.418478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:38.418478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:38.418478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:38.418478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:38.418478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:38.419660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:38.419660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:38.419660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:38.419660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:38.419660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:38.419660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:38.419660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:38.419660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:38.419660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:38.419660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:38.419660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:38.419660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:38.419660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:38.419660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:38.419660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:38.419660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:38.419660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:38.420742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:38.420742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:38.420742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:38.420742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:38.422810 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 311/603] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:38.424183 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c9fc7-8289-478d-aedf-33b8d8abec93 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:38.450879 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318adc9-075a-4740-9c13-22f72f009ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:38.451920 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318adc9-075a-4740-9c13-22f72f009ddc None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:38.452312 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318adc9-075a-4740-9c13-22f72f009ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:38.452693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318adc9-075a-4740-9c13-22f72f009ddc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:38.464879 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318adc9-075a-4740-9c13-22f72f009ddc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:38.473424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-9318adc9-075a-4740-9c13-22f72f009ddc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:38.473424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:38.473424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:38.473424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:38.473424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:38.473424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:38.473424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:38.473424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:38.473424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:38.473424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:38.473424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:38.473424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:38.473424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:38.473424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:38.473424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:38.473424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:38.474379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:38.474379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:38.474379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:38.474379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:38.474379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:38.474379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:38.474379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:38.474379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:38.474379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:38.474379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:38.474379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:38.474379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:38.474379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:38.474379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:38.474379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:38.474379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:38.475216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:38.475216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:38.475216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:38.475216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:38.475216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:38.475216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:38.475216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:38.475216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:38.475216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:38.475216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:38.475216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:38.475216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:38.475216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 293/604] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:38.476018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318adc9-075a-4740-9c13-22f72f009ddc None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:38.522435 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:38.570837 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c9fc7-8289-478d-aedf-33b8d8abec93 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:38.571135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:38.611132 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ab9c3-30bd-4da0-afd9-44fd8cce0b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:38.612109 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ab9c3-30bd-4da0-afd9-44fd8cce0b37 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:38.613079 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ab9c3-30bd-4da0-afd9-44fd8cce0b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:38.613475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ab9c3-30bd-4da0-afd9-44fd8cce0b37 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:38.614374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ab9c3-30bd-4da0-afd9-44fd8cce0b37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:38.614789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ab9c3-30bd-4da0-afd9-44fd8cce0b37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:38.616619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ab9c3-30bd-4da0-afd9-44fd8cce0b37 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:38.631108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318adc9-075a-4740-9c13-22f72f009ddc None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:38.634689 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 312/605] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:38.635950 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ab9c3-30bd-4da0-afd9-44fd8cce0b37 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:38.680826 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd3458-162e-46dc-90e0-1b9974f0f79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:38.682282 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd3458-162e-46dc-90e0-1b9974f0f79a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:38.682729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd3458-162e-46dc-90e0-1b9974f0f79a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:38.683159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd3458-162e-46dc-90e0-1b9974f0f79a None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:38.684590 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd3458-162e-46dc-90e0-1b9974f0f79a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:38.685066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd3458-162e-46dc-90e0-1b9974f0f79a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:38.687413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd3458-162e-46dc-90e0-1b9974f0f79a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:38.706399 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 294/606] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:38.707592 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd3458-162e-46dc-90e0-1b9974f0f79a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:38.747558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ab9c3-30bd-4da0-afd9-44fd8cce0b37 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:38.764650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:38.784762 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649383f6-95ef-4fa0-abb6-3cc55154fe58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:38.786027 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649383f6-95ef-4fa0-abb6-3cc55154fe58 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:38.786027 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649383f6-95ef-4fa0-abb6-3cc55154fe58 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:38.786489 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649383f6-95ef-4fa0-abb6-3cc55154fe58 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:38.787413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649383f6-95ef-4fa0-abb6-3cc55154fe58 None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:38.787830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649383f6-95ef-4fa0-abb6-3cc55154fe58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:38.789311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649383f6-95ef-4fa0-abb6-3cc55154fe58 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:38.804143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 313/607] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:49:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:38.804143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649383f6-95ef-4fa0-abb6-3cc55154fe58 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:38.826303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd3458-162e-46dc-90e0-1b9974f0f79a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:38.847637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:38.871017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5dc6b6-c167-42b0-adc3-8dcdd9936ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:38.871950 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dc6b6-c167-42b0-adc3-8dcdd9936ec5 None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:38.872238 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dc6b6-c167-42b0-adc3-8dcdd9936ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:38.872520 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dc6b6-c167-42b0-adc3-8dcdd9936ec5 None admin] PATH_INFO: `/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/e5d76eae62b54521a95f36284118fada/roles/24846e82841b4b05b4d9e9d193a98868` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:38.890585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5dc6b6-c167-42b0-adc3-8dcdd9936ec5 None admin] RBAC: Authorizing `identity:create_grant(project_id=6003dba8ddd247e7842d28ea7ca5238c, user_id=e5d76eae62b54521a95f36284118fada, role_id=24846e82841b4b05b4d9e9d193a98868)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:38.892477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5dc6b6-c167-42b0-adc3-8dcdd9936ec5 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:38.929092 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649383f6-95ef-4fa0-abb6-3cc55154fe58 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:38.964690 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9994dfb-ebde-4e94-aaa4-da9d9c775232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:38.965522 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9994dfb-ebde-4e94-aaa4-da9d9c775232 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:38.965814 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9994dfb-ebde-4e94-aaa4-da9d9c775232 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:38.966111 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9994dfb-ebde-4e94-aaa4-da9d9c775232 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:38.973800 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9994dfb-ebde-4e94-aaa4-da9d9c775232 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:38.975416 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9994dfb-ebde-4e94-aaa4-da9d9c775232 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:38.991013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-d9994dfb-ebde-4e94-aaa4-da9d9c775232 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:38.991013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:38.991013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:38.991013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:38.991013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:38.991013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:38.991013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:38.991013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:38.991013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:38.991013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:38.991013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:38.991013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:38.991013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:38.991013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:38.991013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:38.992351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:38.992351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:38.992351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:38.992351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:38.992351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:38.992351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:38.992351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:38.992351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:38.992351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:38.992351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:38.992351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:38.992351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:38.992351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:38.992351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:38.992351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:38.992351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:38.992351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:38.993224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:38.993224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:38.993224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:38.993224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:38.993224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:38.993224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:38.993224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:38.993224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:38.994171 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 295/609] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:49:38 2022] PUT /identity/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/e5d76eae62b54521a95f36284118fada/roles/24846e82841b4b05b4d9e9d193a98868 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:49:38.995492 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5dc6b6-c167-42b0-adc3-8dcdd9936ec5 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:38.997447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 314/609] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:38.998403 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9994dfb-ebde-4e94-aaa4-da9d9c775232 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:39.092443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:39.202536 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5dc6b6-c167-42b0-adc3-8dcdd9936ec5 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:39.207489 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9994dfb-ebde-4e94-aaa4-da9d9c775232 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:39.244433 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d90ae-1b4e-487a-adcf-bccf8f7be094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:39.245072 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be57a67-6954-429d-89ff-1c1bdcdf7d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:39.245295 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d90ae-1b4e-487a-adcf-bccf8f7be094 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:39.245570 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d90ae-1b4e-487a-adcf-bccf8f7be094 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:39.245737 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be57a67-6954-429d-89ff-1c1bdcdf7d8d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:39.245832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d90ae-1b4e-487a-adcf-bccf8f7be094 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:39.246028 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be57a67-6954-429d-89ff-1c1bdcdf7d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:39.246309 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be57a67-6954-429d-89ff-1c1bdcdf7d8d None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:39.247319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be57a67-6954-429d-89ff-1c1bdcdf7d8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:39.247738 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be57a67-6954-429d-89ff-1c1bdcdf7d8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:39.249289 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be57a67-6954-429d-89ff-1c1bdcdf7d8d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:39.258052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d90ae-1b4e-487a-adcf-bccf8f7be094 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:39.259690 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d90ae-1b4e-487a-adcf-bccf8f7be094 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:39.262830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 315/610] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:39.276177 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be57a67-6954-429d-89ff-1c1bdcdf7d8d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:39.278718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-583d90ae-1b4e-487a-adcf-bccf8f7be094 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:39.278718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:39.278718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:39.278718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:39.278718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:39.278718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:39.278718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:39.278718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:39.278718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:39.278718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:39.278718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:39.278718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:39.278718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:39.278718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:39.278718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:39.279638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:39.279638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:39.279638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:39.279638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:39.279638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:39.279638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:39.279638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:39.279638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:39.279638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:39.279638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:39.279638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:39.279638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:39.279638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:39.279638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:39.279638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:39.279638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:39.279638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:39.280503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:39.280503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:39.280503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:39.280503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:39.280503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 296/611] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:39.287649 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d90ae-1b4e-487a-adcf-bccf8f7be094 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:39.331849 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:39.465251 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d90ae-1b4e-487a-adcf-bccf8f7be094 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:39.476990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:39.484214 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be57a67-6954-429d-89ff-1c1bdcdf7d8d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:39.527150 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda1e46-be6a-4078-ae95-ca629f889ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:39.528236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda1e46-be6a-4078-ae95-ca629f889ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:39.528820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda1e46-be6a-4078-ae95-ca629f889ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:39.529310 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda1e46-be6a-4078-ae95-ca629f889ee5 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:39.530280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda1e46-be6a-4078-ae95-ca629f889ee5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:39.530767 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd228196-5c4b-450e-bb64-77390e855efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:39.531055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda1e46-be6a-4078-ae95-ca629f889ee5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:39.531684 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd228196-5c4b-450e-bb64-77390e855efe None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:39.531972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd228196-5c4b-450e-bb64-77390e855efe None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:39.532250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd228196-5c4b-450e-bb64-77390e855efe None admin] PATH_INFO: `/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/74b8c75b9cce442b8c430fab6a3fea17/roles/24846e82841b4b05b4d9e9d193a98868` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:39.533281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda1e46-be6a-4078-ae95-ca629f889ee5 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:39.555504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 297/612] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:39.561896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda1e46-be6a-4078-ae95-ca629f889ee5 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:39.566609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd228196-5c4b-450e-bb64-77390e855efe None admin] RBAC: Authorizing `identity:create_grant(project_id=6003dba8ddd247e7842d28ea7ca5238c, user_id=74b8c75b9cce442b8c430fab6a3fea17, role_id=24846e82841b4b05b4d9e9d193a98868)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:39.568440 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd228196-5c4b-450e-bb64-77390e855efe None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:39.632705 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 316/613] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:49:39 2022] PUT /identity/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/74b8c75b9cce442b8c430fab6a3fea17/roles/24846e82841b4b05b4d9e9d193a98868 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:49:39.676938 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:39.712674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda1e46-be6a-4078-ae95-ca629f889ee5 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:39.762870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8544bee-7ce5-44f7-90ac-7537f980b6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:39.764116 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8544bee-7ce5-44f7-90ac-7537f980b6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:39.764613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8544bee-7ce5-44f7-90ac-7537f980b6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:39.765103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8544bee-7ce5-44f7-90ac-7537f980b6e0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:39.774887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8544bee-7ce5-44f7-90ac-7537f980b6e0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:39.777104 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8544bee-7ce5-44f7-90ac-7537f980b6e0 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:39.793170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-f8544bee-7ce5-44f7-90ac-7537f980b6e0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 18:49:39.793170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:39.793170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:39.793170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:39.793170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:39.793170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:39.793170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:39.793170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:39.793170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:39.793170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:39.793170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:39.793170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:39.793170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:39.793170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:39.793170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:39.793170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:39.794152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:39.794152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:39.794152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:39.794152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:39.794152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:39.794152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:39.794152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:39.794152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:39.794152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:39.794152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:39.794152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:39.794152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:39.794152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:39.794152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:39.794152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:39.794152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:39.794152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:39.794152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:39.795108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:39.795108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:39.795108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:39.795108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:39.795108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:39.795108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:39.795108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:39.795108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:39.795108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 18:49:39.795108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:39.797926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 298/614] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:49:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:39.803932 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd228196-5c4b-450e-bb64-77390e855efe None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:39.915623 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd228196-5c4b-450e-bb64-77390e855efe None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:39.952017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f02f2-7056-4e04-8cd7-54d054078a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:39.953079 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f02f2-7056-4e04-8cd7-54d054078a41 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:39.953586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f02f2-7056-4e04-8cd7-54d054078a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:39.954066 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f02f2-7056-4e04-8cd7-54d054078a41 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:39.955130 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f02f2-7056-4e04-8cd7-54d054078a41 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:39.955698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f02f2-7056-4e04-8cd7-54d054078a41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:39.958119 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f02f2-7056-4e04-8cd7-54d054078a41 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:39.978945 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 317/615] 10.222.0.69 () {56 vars in 1135 bytes} [Thu Jul 14 18:49:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:39.984798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8544bee-7ce5-44f7-90ac-7537f980b6e0 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:40.039082 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086f7d7-b33d-418b-b12c-8a1a4d176fab None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:40.040331 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086f7d7-b33d-418b-b12c-8a1a4d176fab None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:40.040331 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086f7d7-b33d-418b-b12c-8a1a4d176fab None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:40.041791 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 318/616] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:40.047953 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01303ad4-7d72-49d9-8efa-af5847922c4f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:40.048415 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01303ad4-7d72-49d9-8efa-af5847922c4f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:40.048866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01303ad4-7d72-49d9-8efa-af5847922c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:40.063494 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:40.077031 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-01303ad4-7d72-49d9-8efa-af5847922c4f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:40.078327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-01303ad4-7d72-49d9-8efa-af5847922c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:40.141507 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8544bee-7ce5-44f7-90ac-7537f980b6e0 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:40.179999 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4d4e4-40cf-436a-b70e-b92167c0717f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:40.180911 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4d4e4-40cf-436a-b70e-b92167c0717f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:40.181296 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4d4e4-40cf-436a-b70e-b92167c0717f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:40.181580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4d4e4-40cf-436a-b70e-b92167c0717f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:40.186146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 319/617] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:49:40.192938 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4d4e4-40cf-436a-b70e-b92167c0717f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:40.194400 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4d4e4-40cf-436a-b70e-b92167c0717f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:40.195595 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e0c00-3c88-45a3-904e-e9408e5bed24 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:40.195904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e0c00-3c88-45a3-904e-e9408e5bed24 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:40.196160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:40.197222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e0c00-3c88-45a3-904e-e9408e5bed24 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:40.198893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 320/618] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:40 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:40.208988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d60424-1f78-4510-8746-2d98ae1c8992 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:40.209275 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d60424-1f78-4510-8746-2d98ae1c8992 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:40.209555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d60424-1f78-4510-8746-2d98ae1c8992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:40.217008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-06f4d4e4-40cf-436a-b70e-b92167c0717f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:40.217008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:40.217008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:40.217008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:40.217008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:40.217008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:40.217008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:40.217008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:40.217008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:40.217008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:40.217008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:40.217008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:40.217008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:40.217008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:40.217008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:40.217870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:40.217870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:40.217870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:40.217870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:40.217870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:40.217870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:40.217870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:40.217870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:40.217870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:40.217870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:40.217870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:40.217870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:40.217870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:40.217870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:40.217870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:40.217870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:40.217870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:40.218963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:40.218963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:40.218963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:40.218963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:40.218963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 299/619] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:40.225977 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4d4e4-40cf-436a-b70e-b92167c0717f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:40.233978 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-65d60424-1f78-4510-8746-2d98ae1c8992 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:40.235189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-65d60424-1f78-4510-8746-2d98ae1c8992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:40.307408 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 321/620] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:49:40.308249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d60424-1f78-4510-8746-2d98ae1c8992 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:40.356529 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4d4e4-40cf-436a-b70e-b92167c0717f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:40.394111 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a8b41-afad-48dc-8313-ddf491d05c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:40.395073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a8b41-afad-48dc-8313-ddf491d05c9b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:40.395396 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a8b41-afad-48dc-8313-ddf491d05c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:40.395798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a8b41-afad-48dc-8313-ddf491d05c9b None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:40.396661 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a8b41-afad-48dc-8313-ddf491d05c9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:40.397082 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a8b41-afad-48dc-8313-ddf491d05c9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:40.398623 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a8b41-afad-48dc-8313-ddf491d05c9b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:40.399797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:40.418146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 300/621] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:40.419537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a8b41-afad-48dc-8313-ddf491d05c9b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:40.450562 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d60424-1f78-4510-8746-2d98ae1c8992 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:40.488733 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:40.490640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca97c8f-8db0-4438-92cd-d18c13f18d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:40.491694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca97c8f-8db0-4438-92cd-d18c13f18d41 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:40.492088 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca97c8f-8db0-4438-92cd-d18c13f18d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:40.492466 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca97c8f-8db0-4438-92cd-d18c13f18d41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:40.504117 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca97c8f-8db0-4438-92cd-d18c13f18d41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:40.512296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-9ca97c8f-8db0-4438-92cd-d18c13f18d41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:40.512296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:40.512296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:40.512296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:40.512296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:40.512296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:40.512296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:40.512296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:40.512296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:40.512296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:40.512296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:40.512296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:40.512296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:40.512296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:40.512296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:40.512296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:40.513413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:40.513413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:40.513413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:40.513413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:40.513413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:40.513413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:40.513413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:40.513413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:40.513413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:40.513413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:40.513413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:40.513413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:40.513413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:40.513413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:40.513413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:40.513413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:40.514468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:40.514468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:40.514468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:40.514468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:40.514468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:40.514468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:40.514468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:40.514468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:40.514468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:40.514468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:40.514468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:40.514468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:40.516583 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 322/622] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:40.517622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca97c8f-8db0-4438-92cd-d18c13f18d41 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:40.557202 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a8b41-afad-48dc-8313-ddf491d05c9b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:40.600098 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c3021-b056-4982-916f-ebfc58aacbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:40.600899 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c3021-b056-4982-916f-ebfc58aacbbe None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:40.601179 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c3021-b056-4982-916f-ebfc58aacbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:40.601451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c3021-b056-4982-916f-ebfc58aacbbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:40.616065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c3021-b056-4982-916f-ebfc58aacbbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:40.625934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-0b7c3021-b056-4982-916f-ebfc58aacbbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:40.625934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:40.625934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:40.625934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:40.625934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:40.625934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:40.625934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:40.625934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:40.625934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:40.625934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:40.625934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:40.625934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:40.625934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:40.625934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:40.625934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:40.625934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:40.626872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:40.626872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:40.626872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:40.626872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:40.626872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:40.626872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:40.626872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:40.626872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:40.626872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:40.626872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:40.626872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:40.626872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:40.626872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:40.626872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:40.626872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:40.626872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:40.627794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:40.627794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:40.627794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:40.627794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:40.627794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:40.627794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:40.627794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:40.627794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:40.627794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:40.627794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:40.627794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:40.627794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:40.627794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 301/623] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:40.628625 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c3021-b056-4982-916f-ebfc58aacbbe None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:40.651158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca97c8f-8db0-4438-92cd-d18c13f18d41 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:40.685525 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:40.697305 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de262e-40bd-485d-b053-0ec80b67d3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:40.698615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de262e-40bd-485d-b053-0ec80b67d3fd None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:40.699075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de262e-40bd-485d-b053-0ec80b67d3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:40.699613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de262e-40bd-485d-b053-0ec80b67d3fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:40.709345 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de262e-40bd-485d-b053-0ec80b67d3fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:40.711569 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de262e-40bd-485d-b053-0ec80b67d3fd None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:40.731650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-46de262e-40bd-485d-b053-0ec80b67d3fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:40.731650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:40.731650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:40.731650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:40.731650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:40.731650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:40.731650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:40.731650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:40.731650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:40.731650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:40.731650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:40.731650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:40.731650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:40.731650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:40.731650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:40.732604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:40.732604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:40.732604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:40.732604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:40.732604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:40.732604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:40.732604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:40.732604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:40.732604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:40.732604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:40.732604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:40.732604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:40.732604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:40.732604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:40.732604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:40.732604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:40.732604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:40.733511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:40.733511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:40.733511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:40.733511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:40.733511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:40.733511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:40.733511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:40.733511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:40.736322 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 323/624] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:40.737279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de262e-40bd-485d-b053-0ec80b67d3fd None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:40.745668 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c3021-b056-4982-916f-ebfc58aacbbe None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:40.756729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:40.782928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2586e-0d85-45f5-94a4-8a7333e4f211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:40.783116 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2586e-0d85-45f5-94a4-8a7333e4f211 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:40.783446 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2586e-0d85-45f5-94a4-8a7333e4f211 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:40.783798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2586e-0d85-45f5-94a4-8a7333e4f211 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:40.786137 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2586e-0d85-45f5-94a4-8a7333e4f211 None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:40.786530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2586e-0d85-45f5-94a4-8a7333e4f211 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:40.788067 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2586e-0d85-45f5-94a4-8a7333e4f211 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:40.797180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 302/625] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:49:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:40.798157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2586e-0d85-45f5-94a4-8a7333e4f211 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:40.868389 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de262e-40bd-485d-b053-0ec80b67d3fd None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:40.915887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc975de-0b5b-44c3-8443-79e74bfb5c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:40.916899 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc975de-0b5b-44c3-8443-79e74bfb5c1c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:40.917011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2586e-0d85-45f5-94a4-8a7333e4f211 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:40.917491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc975de-0b5b-44c3-8443-79e74bfb5c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:40.917953 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc975de-0b5b-44c3-8443-79e74bfb5c1c None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:40.919003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc975de-0b5b-44c3-8443-79e74bfb5c1c None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:40.919605 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc975de-0b5b-44c3-8443-79e74bfb5c1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:40.921203 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc975de-0b5b-44c3-8443-79e74bfb5c1c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:40.930987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 324/626] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:49:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:40.932133 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc975de-0b5b-44c3-8443-79e74bfb5c1c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:40.952318 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f86ac-926d-4870-a7f8-033111d92443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:40.952986 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f86ac-926d-4870-a7f8-033111d92443 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:40.953272 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f86ac-926d-4870-a7f8-033111d92443 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:40.953551 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f86ac-926d-4870-a7f8-033111d92443 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:40.954481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f86ac-926d-4870-a7f8-033111d92443 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:40.954729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f86ac-926d-4870-a7f8-033111d92443 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:40.956845 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f86ac-926d-4870-a7f8-033111d92443 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:40.974225 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 303/627] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:40.975212 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f86ac-926d-4870-a7f8-033111d92443 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:40.992213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:41.009590 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:41.053939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc975de-0b5b-44c3-8443-79e74bfb5c1c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:41.080156 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f86ac-926d-4870-a7f8-033111d92443 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:41.093506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27ac14-b71a-4310-bbbe-2c9360879ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:41.094703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27ac14-b71a-4310-bbbe-2c9360879ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:41.095246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27ac14-b71a-4310-bbbe-2c9360879ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:41.095794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27ac14-b71a-4310-bbbe-2c9360879ab0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:41.107956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c27ac14-b71a-4310-bbbe-2c9360879ab0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:41.109790 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c27ac14-b71a-4310-bbbe-2c9360879ab0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:41.115801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdd9ab-4657-4dc1-91b6-c23be697f271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:41.116651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdd9ab-4657-4dc1-91b6-c23be697f271 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:41.116939 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdd9ab-4657-4dc1-91b6-c23be697f271 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:41.117215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdd9ab-4657-4dc1-91b6-c23be697f271 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:41.127687 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdd9ab-4657-4dc1-91b6-c23be697f271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:41.129659 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdd9ab-4657-4dc1-91b6-c23be697f271 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:41.132269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-8c27ac14-b71a-4310-bbbe-2c9360879ab0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:41.132269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:41.132269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:41.132269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:41.132269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:41.132269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:41.132269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:41.132269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:41.132269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:41.132269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:41.132269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:41.132269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:41.132269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:41.132269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:41.132269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:41.133240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:41.133240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:41.133240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:41.133240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:41.133240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:41.133240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:41.133240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:41.133240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:41.133240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:41.133240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:41.133240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:41.133240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:41.133240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:41.133240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:41.133240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:41.133240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:41.133240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:41.134151 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:41.134151 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:41.134151 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:41.134151 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:41.136574 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 325/628] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:41.137548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27ac14-b71a-4310-bbbe-2c9360879ab0 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:41.148853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-96cdd9ab-4657-4dc1-91b6-c23be697f271 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:41.148853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:41.148853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:41.148853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:41.148853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:41.148853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:41.148853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:41.148853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:41.148853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:41.148853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:41.148853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:41.148853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:41.148853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:41.148853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:41.148853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:41.149764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:41.149764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:41.149764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:41.149764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:41.149764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:41.149764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:41.149764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:41.149764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:41.149764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:41.149764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:41.149764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:41.149764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:41.149764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:41.149764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:41.149764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:41.149764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:41.149764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:41.150667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:41.150667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:41.150667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:41.150667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:41.150667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 304/629] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:41.151649 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdd9ab-4657-4dc1-91b6-c23be697f271 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:41.236726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:41.250941 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:41.252734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdd9ab-4657-4dc1-91b6-c23be697f271 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:41.255455 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27ac14-b71a-4310-bbbe-2c9360879ab0 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:41.289098 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f40ff-a571-4184-a98a-fad2dc03a0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:41.289766 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f40ff-a571-4184-a98a-fad2dc03a0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:41.290055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f40ff-a571-4184-a98a-fad2dc03a0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:41.290317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f40ff-a571-4184-a98a-fad2dc03a0b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:41.291038 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f40ff-a571-4184-a98a-fad2dc03a0b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:41.291373 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f40ff-a571-4184-a98a-fad2dc03a0b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:41.292743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f40ff-a571-4184-a98a-fad2dc03a0b4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:41.293375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ec98e-2276-42e9-b240-8bcdb9dfeb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:41.294452 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ec98e-2276-42e9-b240-8bcdb9dfeb8d None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:41.294905 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ec98e-2276-42e9-b240-8bcdb9dfeb8d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:41.295555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ec98e-2276-42e9-b240-8bcdb9dfeb8d None admin] PATH_INFO: `/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/4368f68724f946eca53084a9ec7ff590/roles/24846e82841b4b05b4d9e9d193a98868` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:41.305378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 305/630] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:41.306190 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f40ff-a571-4184-a98a-fad2dc03a0b4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:41.315064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ec98e-2276-42e9-b240-8bcdb9dfeb8d None admin] RBAC: Authorizing `identity:create_grant(project_id=6003dba8ddd247e7842d28ea7ca5238c, user_id=4368f68724f946eca53084a9ec7ff590, role_id=24846e82841b4b05b4d9e9d193a98868)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:41.317306 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ec98e-2276-42e9-b240-8bcdb9dfeb8d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:41.363868 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 326/631] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:49:41 2022] PUT /identity/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/4368f68724f946eca53084a9ec7ff590/roles/24846e82841b4b05b4d9e9d193a98868 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:49:41.365340 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11915ef-29b5-4284-b794-398f8caf6a1a None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:41.365887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11915ef-29b5-4284-b794-398f8caf6a1a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:41.366249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11915ef-29b5-4284-b794-398f8caf6a1a None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:41.367699 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 327/632] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:41.368576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11915ef-29b5-4284-b794-398f8caf6a1a None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:41.449675 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f40ff-a571-4184-a98a-fad2dc03a0b4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:41.496598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11915ef-29b5-4284-b794-398f8caf6a1a None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:41.500065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e857cb6-8c1f-432d-ac07-537be5d0426c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:41.501948 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e857cb6-8c1f-432d-ac07-537be5d0426c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:41.502346 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e857cb6-8c1f-432d-ac07-537be5d0426c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:41.502745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e857cb6-8c1f-432d-ac07-537be5d0426c None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:41.504276 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e857cb6-8c1f-432d-ac07-537be5d0426c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:41.504662 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e857cb6-8c1f-432d-ac07-537be5d0426c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:41.506170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e857cb6-8c1f-432d-ac07-537be5d0426c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:41.524762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 306/633] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:41.526891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581bb17-ac24-4d08-b6a5-a9a404e82f8f None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:41.527287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581bb17-ac24-4d08-b6a5-a9a404e82f8f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:41.527649 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581bb17-ac24-4d08-b6a5-a9a404e82f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:41.536890 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:41.549381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368c1ef-ce45-43f9-9e80-0a40533cc1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:41.554571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a581bb17-ac24-4d08-b6a5-a9a404e82f8f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:41.554702 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368c1ef-ce45-43f9-9e80-0a40533cc1ba None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:41.554782 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a581bb17-ac24-4d08-b6a5-a9a404e82f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:41.554855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368c1ef-ce45-43f9-9e80-0a40533cc1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:41.554855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368c1ef-ce45-43f9-9e80-0a40533cc1ba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:41.563207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f368c1ef-ce45-43f9-9e80-0a40533cc1ba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:41.564957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f368c1ef-ce45-43f9-9e80-0a40533cc1ba None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:41.580954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-f368c1ef-ce45-43f9-9e80-0a40533cc1ba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 18:49:41.580954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:41.580954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:41.580954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:41.580954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:41.580954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:41.580954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:41.580954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:41.580954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:41.580954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:41.580954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:41.580954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:41.580954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:41.580954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:41.580954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:41.580954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:41.581944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:41.581944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:41.581944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:41.581944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:41.581944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:41.581944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:41.581944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:41.581944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:41.581944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:41.581944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:41.581944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:41.581944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:41.581944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:41.581944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:41.581944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:41.581944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:41.581944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:41.581944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:41.582871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:41.582871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:41.582871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:41.582871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:41.582871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:41.582871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:41.582871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:41.582871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:41.582871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 18:49:41.582871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:41.587050 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 328/634] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:41.587050 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368c1ef-ce45-43f9-9e80-0a40533cc1ba None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:41.599412 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:41.633169 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 307/635] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:41.634173 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581bb17-ac24-4d08-b6a5-a9a404e82f8f None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:41.714036 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368c1ef-ce45-43f9-9e80-0a40533cc1ba None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:41.749512 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ff481-a0f3-4b70-9987-9f9a01420a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:41.750307 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ff481-a0f3-4b70-9987-9f9a01420a61 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:41.750590 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ff481-a0f3-4b70-9987-9f9a01420a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:41.750919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ff481-a0f3-4b70-9987-9f9a01420a61 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:41.758352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ff481-a0f3-4b70-9987-9f9a01420a61 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:41.759989 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ff481-a0f3-4b70-9987-9f9a01420a61 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:41.763751 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581bb17-ac24-4d08-b6a5-a9a404e82f8f None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:41.774787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-be9ff481-a0f3-4b70-9987-9f9a01420a61 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 18:49:41.774787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:41.774787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:41.774787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:41.774787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:41.774787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:41.774787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:41.774787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:41.774787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:41.774787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:41.774787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:41.774787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:41.774787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:41.774787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:41.774787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:41.774787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:41.775801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:41.775801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:41.775801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:41.775801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:41.775801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:41.775801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:41.775801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:41.775801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:41.775801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:41.775801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:41.775801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:41.775801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:41.775801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:41.775801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:41.775801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:41.775801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:41.775801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:41.775801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:41.777612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:41.777612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:41.777612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:41.777612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:41.777612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:41.777612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:41.777612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:41.777612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:41.777612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 18:49:41.777612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:41.777612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 329/636] 10.222.0.69 () {56 vars in 1093 bytes} [Thu Jul 14 18:49:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:41.777612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ff481-a0f3-4b70-9987-9f9a01420a61 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:41.807574 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6409a4-0875-4261-bfdc-6a2ef71f9e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:41.808627 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6409a4-0875-4261-bfdc-6a2ef71f9e5f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:41.809167 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6409a4-0875-4261-bfdc-6a2ef71f9e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:41.809763 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6409a4-0875-4261-bfdc-6a2ef71f9e5f None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:41.810851 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6409a4-0875-4261-bfdc-6a2ef71f9e5f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:41.811419 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6409a4-0875-4261-bfdc-6a2ef71f9e5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:41.813229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6409a4-0875-4261-bfdc-6a2ef71f9e5f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:41.815197 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:41.838085 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 308/637] 10.222.0.69 () {56 vars in 1137 bytes} [Thu Jul 14 18:49:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:41.839584 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6409a4-0875-4261-bfdc-6a2ef71f9e5f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:41.849557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:41.880534 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ff481-a0f3-4b70-9987-9f9a01420a61 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:41.917054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5709a-bfe9-4ed7-8c5e-d61e6be53f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:41.918051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5709a-bfe9-4ed7-8c5e-d61e6be53f00 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:41.918335 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5709a-bfe9-4ed7-8c5e-d61e6be53f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:41.918610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5709a-bfe9-4ed7-8c5e-d61e6be53f00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:41.930592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea5709a-bfe9-4ed7-8c5e-d61e6be53f00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:41.938462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-bea5709a-bfe9-4ed7-8c5e-d61e6be53f00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:41.938462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:41.938462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:41.938462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:41.938462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:41.938462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:41.938462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:41.938462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:41.938462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:41.938462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:41.938462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:41.938462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:41.938462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:41.938462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:41.938462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:41.938462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:41.939692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:41.939692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:41.939692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:41.939692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:41.939692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:41.939692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:41.939692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:41.939692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:41.939692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:41.939692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:41.939692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:41.939692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:41.939692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:41.939692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:41.939692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:41.939692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:41.940727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:41.940727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:41.940727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:41.940727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:41.940727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:41.940727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:41.940727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:41.940727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:41.940727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:41.940727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:41.940727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:41.940727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:41.944490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 330/638] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:41.945123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5709a-bfe9-4ed7-8c5e-d61e6be53f00 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:41.955790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6409a4-0875-4261-bfdc-6a2ef71f9e5f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:41.992381 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd93b29-4643-4b4f-ac08-b2f453f4226f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:41.993148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd93b29-4643-4b4f-ac08-b2f453f4226f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:41.993535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd93b29-4643-4b4f-ac08-b2f453f4226f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:41.993916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd93b29-4643-4b4f-ac08-b2f453f4226f None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:41.994890 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd93b29-4643-4b4f-ac08-b2f453f4226f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:41.995346 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd93b29-4643-4b4f-ac08-b2f453f4226f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:41.996880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd93b29-4643-4b4f-ac08-b2f453f4226f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:42.019519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 309/639] 10.222.0.69 () {56 vars in 1133 bytes} [Thu Jul 14 18:49:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:42.019519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159ab19-bef0-4361-838b-a9929d25988f None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:42.019519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159ab19-bef0-4361-838b-a9929d25988f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:42.019519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159ab19-bef0-4361-838b-a9929d25988f None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:42.019519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 310/640] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:42.019519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159ab19-bef0-4361-838b-a9929d25988f None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:42.062436 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5709a-bfe9-4ed7-8c5e-d61e6be53f00 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:42.073754 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:42.094438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:42.101305 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95d109-8a33-470b-932f-ca15184334d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:42.102073 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95d109-8a33-470b-932f-ca15184334d5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:42.102465 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95d109-8a33-470b-932f-ca15184334d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:42.102846 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95d109-8a33-470b-932f-ca15184334d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:42.126237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159ab19-bef0-4361-838b-a9929d25988f None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:42.161716 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b496f94-f648-44eb-b0f5-8f8ea1a753c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:42.162414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b496f94-f648-44eb-b0f5-8f8ea1a753c9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:42.162812 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b496f94-f648-44eb-b0f5-8f8ea1a753c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:42.163088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b496f94-f648-44eb-b0f5-8f8ea1a753c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:42.163902 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b496f94-f648-44eb-b0f5-8f8ea1a753c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:42.164211 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b496f94-f648-44eb-b0f5-8f8ea1a753c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:42.165584 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b496f94-f648-44eb-b0f5-8f8ea1a753c9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:42.178429 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 311/641] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:49:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:42.178429 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b496f94-f648-44eb-b0f5-8f8ea1a753c9 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:42.181429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95d109-8a33-470b-932f-ca15184334d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:42.182967 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95d109-8a33-470b-932f-ca15184334d5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:42.285142 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b496f94-f648-44eb-b0f5-8f8ea1a753c9 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:42.293717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 331/642] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:42.293717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700e04c-6589-4d19-90d5-9fdb4f472443 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:42.293717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700e04c-6589-4d19-90d5-9fdb4f472443 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:42.293717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700e04c-6589-4d19-90d5-9fdb4f472443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:42.300495 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:42.309209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3700e04c-6589-4d19-90d5-9fdb4f472443 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:42.310474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3700e04c-6589-4d19-90d5-9fdb4f472443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:42.321312 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560132c-647d-42c0-9c5c-b415981034d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:42.322040 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560132c-647d-42c0-9c5c-b415981034d9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:42.322327 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560132c-647d-42c0-9c5c-b415981034d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:42.322605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560132c-647d-42c0-9c5c-b415981034d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:42.335880 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:42.377531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 332/643] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:42.378678 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700e04c-6589-4d19-90d5-9fdb4f472443 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:42.403695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560132c-647d-42c0-9c5c-b415981034d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:42.405517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560132c-647d-42c0-9c5c-b415981034d9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:42.488174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700e04c-6589-4d19-90d5-9fdb4f472443 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:42.516170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 312/644] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:42.517025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560132c-647d-42c0-9c5c-b415981034d9 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:42.525413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1fd23-34d5-4055-9761-991ce362c13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:42.527066 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1fd23-34d5-4055-9761-991ce362c13d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:42.527589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1fd23-34d5-4055-9761-991ce362c13d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:42.528086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1fd23-34d5-4055-9761-991ce362c13d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:42.536160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:42.540731 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1fd23-34d5-4055-9761-991ce362c13d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:42.542371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1fd23-34d5-4055-9761-991ce362c13d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:42.561668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-78c1fd23-34d5-4055-9761-991ce362c13d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:42.561668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:42.561668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:42.561668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:42.561668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:42.561668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:42.561668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:42.561668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:42.561668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:42.561668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:42.561668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:42.561668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:42.561668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:42.561668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:42.561668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:42.562721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:42.562721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:42.562721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:42.562721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:42.562721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:42.562721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:42.562721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:42.562721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:42.562721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:42.562721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:42.562721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:42.562721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:42.562721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:42.562721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:42.562721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:42.562721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:42.562721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:42.569020 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:42.569020 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:42.569020 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:42.569020 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:42.569020 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 333/645] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:42.569020 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1fd23-34d5-4055-9761-991ce362c13d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:42.583887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:42.635679 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560132c-647d-42c0-9c5c-b415981034d9 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:42.673187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1b002-91be-41fa-b297-84f3f1072678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:42.673984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1b002-91be-41fa-b297-84f3f1072678 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:42.674274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1b002-91be-41fa-b297-84f3f1072678 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:42.674556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1b002-91be-41fa-b297-84f3f1072678 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:42.679633 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1fd23-34d5-4055-9761-991ce362c13d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:42.685341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1b002-91be-41fa-b297-84f3f1072678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:42.686795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1b002-91be-41fa-b297-84f3f1072678 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:42.704827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-43d1b002-91be-41fa-b297-84f3f1072678 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:42.704827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:42.704827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:42.704827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:42.704827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:42.704827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:42.704827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:42.704827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:42.704827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:42.704827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:42.704827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:42.704827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:42.704827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:42.704827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:42.704827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:42.705736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:42.705736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:42.705736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:42.705736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:42.705736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:42.705736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:42.705736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:42.705736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:42.705736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:42.705736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:42.705736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:42.705736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:42.705736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:42.705736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:42.705736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:42.705736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:42.705736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:42.707483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:42.707483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:42.707483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:42.707483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:42.707483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 313/646] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:42.707483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1b002-91be-41fa-b297-84f3f1072678 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:42.723619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac243db-8ca1-4d72-a17f-26460910fa73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:42.724417 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac243db-8ca1-4d72-a17f-26460910fa73 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:42.724701 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac243db-8ca1-4d72-a17f-26460910fa73 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:42.724979 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac243db-8ca1-4d72-a17f-26460910fa73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:42.735702 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac243db-8ca1-4d72-a17f-26460910fa73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:42.744059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-4ac243db-8ca1-4d72-a17f-26460910fa73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:42.744059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:42.744059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:42.744059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:42.744059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:42.744059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:42.744059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:42.744059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:42.744059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:42.744059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:42.744059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:42.744059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:42.744059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:42.744059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:42.744059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:42.744059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:42.745014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:42.745014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:42.745014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:42.745014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:42.745014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:42.745014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:42.745014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:42.745014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:42.745014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:42.745014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:42.745014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:42.745014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:42.745014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:42.745014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:42.745014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:42.745014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:42.745842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:42.745842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:42.745842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:42.745842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:42.745842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:42.745842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:42.745842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:42.745842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:42.745842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:42.745842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:42.745842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:42.745842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:42.745842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 334/647] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:42.746536 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac243db-8ca1-4d72-a17f-26460910fa73 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:42.773063 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:42.811671 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1b002-91be-41fa-b297-84f3f1072678 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:42.827200 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:42.845287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d7dc0-19eb-4de4-9892-92df3194e5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:42.846078 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d7dc0-19eb-4de4-9892-92df3194e5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:42.846362 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d7dc0-19eb-4de4-9892-92df3194e5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:42.846654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d7dc0-19eb-4de4-9892-92df3194e5b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:42.857300 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d7dc0-19eb-4de4-9892-92df3194e5b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:42.858753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d7dc0-19eb-4de4-9892-92df3194e5b4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:42.864304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac243db-8ca1-4d72-a17f-26460910fa73 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:42.878750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-054d7dc0-19eb-4de4-9892-92df3194e5b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:42.878750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:42.878750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:42.878750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:42.878750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:42.878750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:42.878750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:42.878750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:42.878750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:42.878750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:42.878750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:42.878750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:42.878750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:42.878750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:42.878750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:42.879693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:42.879693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:42.879693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:42.879693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:42.879693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:42.879693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:42.879693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:42.879693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:42.879693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:42.879693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:42.879693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:42.879693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:42.879693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:42.879693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:42.879693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:42.879693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:42.879693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:42.880603 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:42.880603 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:42.880603 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:42.880603 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:42.880603 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 314/648] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:42.881646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d7dc0-19eb-4de4-9892-92df3194e5b4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:42.906406 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29e804-5c47-4659-b130-32e70961e624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:42.907177 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29e804-5c47-4659-b130-32e70961e624 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:42.907416 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29e804-5c47-4659-b130-32e70961e624 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:42.907700 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29e804-5c47-4659-b130-32e70961e624 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:42.908424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29e804-5c47-4659-b130-32e70961e624 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:42.908739 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29e804-5c47-4659-b130-32e70961e624 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:42.910245 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29e804-5c47-4659-b130-32e70961e624 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:42.922313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 335/649] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:42.923120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29e804-5c47-4659-b130-32e70961e624 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:42.991907 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d7dc0-19eb-4de4-9892-92df3194e5b4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:43.009263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:43.028526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b9a76-029c-45c2-b0e6-c98a93d969b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:43.029199 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b9a76-029c-45c2-b0e6-c98a93d969b7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:43.029483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b9a76-029c-45c2-b0e6-c98a93d969b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:43.029759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b9a76-029c-45c2-b0e6-c98a93d969b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:43.030476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b9a76-029c-45c2-b0e6-c98a93d969b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:43.030773 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b9a76-029c-45c2-b0e6-c98a93d969b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:43.032172 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b9a76-029c-45c2-b0e6-c98a93d969b7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:43.040616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29e804-5c47-4659-b130-32e70961e624 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:43.047592 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 315/650] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:43.049103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b9a76-029c-45c2-b0e6-c98a93d969b7 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:43.070609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:43.075244 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d30dd-ee8c-4f2b-b26b-deda978eb54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:43.076052 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d30dd-ee8c-4f2b-b26b-deda978eb54a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:43.076256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d30dd-ee8c-4f2b-b26b-deda978eb54a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:43.076601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d30dd-ee8c-4f2b-b26b-deda978eb54a None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:43.077266 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d30dd-ee8c-4f2b-b26b-deda978eb54a None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:43.077576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d30dd-ee8c-4f2b-b26b-deda978eb54a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:43.078949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d30dd-ee8c-4f2b-b26b-deda978eb54a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:43.087001 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 336/651] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:49:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:43.087830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d30dd-ee8c-4f2b-b26b-deda978eb54a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:43.167655 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b9a76-029c-45c2-b0e6-c98a93d969b7 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:43.195044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d30dd-ee8c-4f2b-b26b-deda978eb54a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:43.205588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef587207-43cb-47ec-9481-ebaca61f0f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:43.206285 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef587207-43cb-47ec-9481-ebaca61f0f0f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:43.206573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef587207-43cb-47ec-9481-ebaca61f0f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:43.206854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef587207-43cb-47ec-9481-ebaca61f0f0f None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:43.207718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef587207-43cb-47ec-9481-ebaca61f0f0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:43.208031 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef587207-43cb-47ec-9481-ebaca61f0f0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:43.209540 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef587207-43cb-47ec-9481-ebaca61f0f0f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:43.222370 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 316/652] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:43.223293 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef587207-43cb-47ec-9481-ebaca61f0f0f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:43.233451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f10008-9f70-4ab2-84f4-a952f67847d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:43.234487 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f10008-9f70-4ab2-84f4-a952f67847d7 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:43.234779 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f10008-9f70-4ab2-84f4-a952f67847d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:43.235061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f10008-9f70-4ab2-84f4-a952f67847d7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:43.244306 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f10008-9f70-4ab2-84f4-a952f67847d7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:43.245788 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f10008-9f70-4ab2-84f4-a952f67847d7 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:43.260541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-a1f10008-9f70-4ab2-84f4-a952f67847d7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 18:49:43.260541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:43.260541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:43.260541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:43.260541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:43.260541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:43.260541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:43.260541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:43.260541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:43.260541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:43.260541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:43.260541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:43.260541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:43.260541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:43.260541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:43.260541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:43.261464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:43.261464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:43.261464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:43.261464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:43.261464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:43.261464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:43.261464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:43.261464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:43.261464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:43.261464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:43.261464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:43.261464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:43.261464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:43.261464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:43.261464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:43.261464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:43.261464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:43.261464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:43.267087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:43.267087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:43.267087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:43.267087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:43.267087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:43.267087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:43.267087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:43.267087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:43.267087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 18:49:43.267087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:43.267087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 337/653] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:43.267087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f10008-9f70-4ab2-84f4-a952f67847d7 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:43.269135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:43.322411 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:43.346267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef587207-43cb-47ec-9481-ebaca61f0f0f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:43.369353 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f10008-9f70-4ab2-84f4-a952f67847d7 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:43.383029 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ecf6b-7c6e-4a6a-9f23-5a1083ef8439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:43.383862 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ecf6b-7c6e-4a6a-9f23-5a1083ef8439 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:43.384149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ecf6b-7c6e-4a6a-9f23-5a1083ef8439 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:43.384733 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ecf6b-7c6e-4a6a-9f23-5a1083ef8439 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:43.391488 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ecf6b-7c6e-4a6a-9f23-5a1083ef8439 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:43.392932 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ecf6b-7c6e-4a6a-9f23-5a1083ef8439 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:43.405809 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57163afa-dccf-4da0-be03-e4d23af6535f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:43.406615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57163afa-dccf-4da0-be03-e4d23af6535f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:43.406904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57163afa-dccf-4da0-be03-e4d23af6535f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:43.407203 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57163afa-dccf-4da0-be03-e4d23af6535f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:43.409303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-d12ecf6b-7c6e-4a6a-9f23-5a1083ef8439 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:43.409303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:43.409303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:43.409303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:43.409303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:43.409303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:43.409303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:43.409303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:43.409303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:43.409303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:43.409303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:43.409303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:43.409303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:43.409303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:43.409303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:43.410149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:43.410149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:43.410149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:43.410149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:43.410149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:43.410149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:43.410149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:43.410149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:43.410149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:43.410149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:43.410149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:43.410149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:43.410149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:43.410149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:43.410149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:43.410149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:43.410149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:43.411034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:43.411034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:43.411034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:43.411034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:43.411034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:43.411034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:43.411034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:43.411034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:43.415023 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 317/654] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:43.416093 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ecf6b-7c6e-4a6a-9f23-5a1083ef8439 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:43.419888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57163afa-dccf-4da0-be03-e4d23af6535f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:43.421558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57163afa-dccf-4da0-be03-e4d23af6535f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:43.440222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-57163afa-dccf-4da0-be03-e4d23af6535f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:43.440222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:43.440222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:43.440222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:43.440222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:43.440222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:43.440222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:43.440222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:43.440222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:43.440222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:43.440222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:43.440222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:43.440222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:43.440222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:43.440222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:43.441904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:43.441904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:43.441904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:43.441904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:43.441904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:43.441904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:43.441904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:43.441904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:43.441904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:43.441904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:43.441904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:43.441904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:43.441904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:43.441904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:43.441904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:43.441904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:43.441904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:43.442858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:43.442858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:43.442858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:43.442858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:43.444193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 338/655] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:43.448308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57163afa-dccf-4da0-be03-e4d23af6535f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:43.512058 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:43.517545 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ecf6b-7c6e-4a6a-9f23-5a1083ef8439 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:43.549155 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57163afa-dccf-4da0-be03-e4d23af6535f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:43.551634 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab66ce-b753-4f75-a2a3-a4a8c3c68a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:43.552428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab66ce-b753-4f75-a2a3-a4a8c3c68a52 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:43.552715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab66ce-b753-4f75-a2a3-a4a8c3c68a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:43.552994 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab66ce-b753-4f75-a2a3-a4a8c3c68a52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:43.556462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:43.559678 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab66ce-b753-4f75-a2a3-a4a8c3c68a52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:43.561122 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab66ce-b753-4f75-a2a3-a4a8c3c68a52 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:43.574705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-89ab66ce-b753-4f75-a2a3-a4a8c3c68a52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:43.574705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:43.574705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:43.574705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:43.574705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:43.574705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:43.574705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:43.574705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:43.574705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:43.574705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:43.574705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:43.574705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:43.574705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:43.574705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:43.574705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:43.575609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:43.575609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:43.575609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:43.575609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:43.575609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:43.575609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:43.575609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:43.575609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:43.575609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:43.575609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:43.575609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:43.575609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:43.575609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:43.575609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:43.575609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:43.575609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:43.575609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:43.576464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:43.576464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:43.576464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:43.576464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:43.576464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:43.576464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:43.576464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:43.576464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:43.579315 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 318/656] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:43.580295 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab66ce-b753-4f75-a2a3-a4a8c3c68a52 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:43.583894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc56fd-31ff-4980-958d-a0c759b38ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:43.584572 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc56fd-31ff-4980-958d-a0c759b38ebe None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:43.584861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc56fd-31ff-4980-958d-a0c759b38ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:43.585144 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc56fd-31ff-4980-958d-a0c759b38ebe None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:43.586033 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc56fd-31ff-4980-958d-a0c759b38ebe None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:43.586606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc56fd-31ff-4980-958d-a0c759b38ebe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:43.588556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc56fd-31ff-4980-958d-a0c759b38ebe None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:43.606892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 339/657] 10.222.0.69 () {56 vars in 1137 bytes} [Thu Jul 14 18:49:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:43.607730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc56fd-31ff-4980-958d-a0c759b38ebe None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:43.681369 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab66ce-b753-4f75-a2a3-a4a8c3c68a52 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:43.704852 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc56fd-31ff-4980-958d-a0c759b38ebe None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:43.714988 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08837e9d-792f-41fe-ac54-c1df3726e1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:43.715675 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08837e9d-792f-41fe-ac54-c1df3726e1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:43.715961 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08837e9d-792f-41fe-ac54-c1df3726e1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:43.716236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08837e9d-792f-41fe-ac54-c1df3726e1f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:43.717014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08837e9d-792f-41fe-ac54-c1df3726e1f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:43.717323 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08837e9d-792f-41fe-ac54-c1df3726e1f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:43.718751 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08837e9d-792f-41fe-ac54-c1df3726e1f2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:43.730878 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 319/658] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:43.732122 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23582ad6-de68-481f-baed-10650d96c16d None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:43.732409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23582ad6-de68-481f-baed-10650d96c16d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:43.732683 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23582ad6-de68-481f-baed-10650d96c16d None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:43.733897 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 320/659] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:43.734579 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23582ad6-de68-481f-baed-10650d96c16d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:43.744135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a8eb7-d247-4aca-8fac-61c41209a8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:43.744800 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a8eb7-d247-4aca-8fac-61c41209a8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:43.745082 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a8eb7-d247-4aca-8fac-61c41209a8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:43.745354 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a8eb7-d247-4aca-8fac-61c41209a8c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:43.746060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a8eb7-d247-4aca-8fac-61c41209a8c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:43.746365 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a8eb7-d247-4aca-8fac-61c41209a8c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:43.747772 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a8eb7-d247-4aca-8fac-61c41209a8c8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:43.752126 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:43.760124 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 340/660] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:43.760843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a8eb7-d247-4aca-8fac-61c41209a8c8 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:43.787538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:43.840177 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23582ad6-de68-481f-baed-10650d96c16d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:43.859618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a8eb7-d247-4aca-8fac-61c41209a8c8 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:43.873548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63abec-06ad-4d57-b000-38d210dc7ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:43.874291 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63abec-06ad-4d57-b000-38d210dc7ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:43.874578 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63abec-06ad-4d57-b000-38d210dc7ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:43.874857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63abec-06ad-4d57-b000-38d210dc7ab9 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:43.875689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63abec-06ad-4d57-b000-38d210dc7ab9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:43.876030 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63abec-06ad-4d57-b000-38d210dc7ab9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:43.877468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63abec-06ad-4d57-b000-38d210dc7ab9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:43.889558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 321/661] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:43.890128 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63abec-06ad-4d57-b000-38d210dc7ab9 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:43.892168 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026087f-1934-487a-bf1e-4677755362e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:43.892814 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026087f-1934-487a-bf1e-4677755362e8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:43.893103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026087f-1934-487a-bf1e-4677755362e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:43.893381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026087f-1934-487a-bf1e-4677755362e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:43.961007 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026087f-1934-487a-bf1e-4677755362e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:43.962801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026087f-1934-487a-bf1e-4677755362e8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:43.970511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:43.985693 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:43.992667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63abec-06ad-4d57-b000-38d210dc7ab9 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:44.028432 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aae21c-2c38-4841-ae84-3c30c7fce211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:44.029083 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aae21c-2c38-4841-ae84-3c30c7fce211 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:44.029367 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aae21c-2c38-4841-ae84-3c30c7fce211 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:44.029650 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aae21c-2c38-4841-ae84-3c30c7fce211 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:44.030468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aae21c-2c38-4841-ae84-3c30c7fce211 None admin] RBAC: Adding query filter params (role.id=24846e82841b4b05b4d9e9d193a98868, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=31b09d72314349c6be0f906484895608) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:44.030786 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aae21c-2c38-4841-ae84-3c30c7fce211 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:44.032239 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aae21c-2c38-4841-ae84-3c30c7fce211 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:44.038055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 322/662] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:49:43 2022] GET /identity/v3/role_assignments?user.id=31b09d72314349c6be0f906484895608&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=24846e82841b4b05b4d9e9d193a98868 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:44.039056 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5900a87-1bb7-4168-bccc-23c8f8caaa23 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:44.039370 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5900a87-1bb7-4168-bccc-23c8f8caaa23 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:44.039651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5900a87-1bb7-4168-bccc-23c8f8caaa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:44.068490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 341/663] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:44.068701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b5900a87-1bb7-4168-bccc-23c8f8caaa23 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:44.068795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026087f-1934-487a-bf1e-4677755362e8 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:44.068880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b5900a87-1bb7-4168-bccc-23c8f8caaa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:44.147482 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 323/664] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:44.147482 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5900a87-1bb7-4168-bccc-23c8f8caaa23 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:44.185141 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026087f-1934-487a-bf1e-4677755362e8 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:44.217461 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:44.229541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6e1e0-0017-4635-af6d-e166b5bf6967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:44.230838 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6e1e0-0017-4635-af6d-e166b5bf6967 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:44.236993 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6e1e0-0017-4635-af6d-e166b5bf6967 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:44.236993 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6e1e0-0017-4635-af6d-e166b5bf6967 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:44.237169 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:44.241803 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6e1e0-0017-4635-af6d-e166b5bf6967 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:44.243650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6e1e0-0017-4635-af6d-e166b5bf6967 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:44.260367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-bdc6e1e0-0017-4635-af6d-e166b5bf6967 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 18:49:44.260367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:44.260367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:44.260367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:44.260367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:44.260367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:44.260367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:44.260367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:44.260367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:44.260367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:44.260367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:44.260367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:44.260367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:44.260367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:44.260367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:44.260367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:44.261601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:44.261601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:44.261601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:44.261601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:44.261601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:44.261601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:44.261601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:44.261601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:44.261601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:44.261601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:44.261601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:44.261601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:44.261601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:44.261601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:44.261601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:44.261601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:44.261601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:44.261601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:44.262549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:44.262549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:44.262549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:44.262549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:44.262549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:44.262549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:44.262549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:44.262549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:44.262549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 18:49:44.262549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:44.263705 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 342/665] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:44.264131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6e1e0-0017-4635-af6d-e166b5bf6967 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:44.269809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5900a87-1bb7-4168-bccc-23c8f8caaa23 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:44.307500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b35310-9a6c-44a5-ae92-0dfa1b279088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:44.308532 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b35310-9a6c-44a5-ae92-0dfa1b279088 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:44.308827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b35310-9a6c-44a5-ae92-0dfa1b279088 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:44.309435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b35310-9a6c-44a5-ae92-0dfa1b279088 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:44.310283 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b35310-9a6c-44a5-ae92-0dfa1b279088 None admin] RBAC: Adding query filter params (role.id=24846e82841b4b05b4d9e9d193a98868, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=8d27d6df5cb64697adebd961dfbec324) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:44.310605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b35310-9a6c-44a5-ae92-0dfa1b279088 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:44.312139 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b35310-9a6c-44a5-ae92-0dfa1b279088 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:44.327195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 324/666] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:49:44 2022] GET /identity/v3/role_assignments?user.id=8d27d6df5cb64697adebd961dfbec324&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=24846e82841b4b05b4d9e9d193a98868 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:44.327195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b35310-9a6c-44a5-ae92-0dfa1b279088 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:44.389303 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6e1e0-0017-4635-af6d-e166b5bf6967 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:44.430428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868054a6-c612-4b15-bf9c-d0dcd37db1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:44.431341 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868054a6-c612-4b15-bf9c-d0dcd37db1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:44.431703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868054a6-c612-4b15-bf9c-d0dcd37db1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:44.432059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868054a6-c612-4b15-bf9c-d0dcd37db1d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:44.444961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868054a6-c612-4b15-bf9c-d0dcd37db1d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:44.446573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868054a6-c612-4b15-bf9c-d0dcd37db1d8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:44.464036 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b35310-9a6c-44a5-ae92-0dfa1b279088 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:44.466060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-868054a6-c612-4b15-bf9c-d0dcd37db1d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:44.466060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:44.466060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:44.466060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:44.466060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:44.466060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:44.466060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:44.466060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:44.466060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:44.466060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:44.466060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:44.466060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:44.466060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:44.466060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:44.466060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:44.466957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:44.466957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:44.466957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:44.466957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:44.466957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:44.466957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:44.466957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:44.466957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:44.466957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:44.466957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:44.466957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:44.466957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:44.466957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:44.466957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:44.466957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:44.466957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:44.466957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:44.467909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:44.467909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:44.467909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:44.467909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:44.467909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 343/667] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:44.468835 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868054a6-c612-4b15-bf9c-d0dcd37db1d8 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:44.512446 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc016a6-a179-4491-a615-cd272c23527d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:44.514325 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc016a6-a179-4491-a615-cd272c23527d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:44.514807 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc016a6-a179-4491-a615-cd272c23527d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:44.515134 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc016a6-a179-4491-a615-cd272c23527d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:44.515543 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:44.517465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:44.530895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc016a6-a179-4491-a615-cd272c23527d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:44.539790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-ddc016a6-a179-4491-a615-cd272c23527d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:44.539790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:44.539790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:44.539790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:44.539790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:44.539790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:44.539790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:44.539790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:44.539790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:44.539790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:44.539790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:44.539790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:44.539790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:44.539790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:44.539790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:44.539790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:44.540850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:44.540850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:44.540850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:44.540850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:44.540850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:44.540850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:44.540850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:44.540850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:44.540850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:44.540850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:44.540850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:44.540850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:44.540850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:44.540850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:44.540850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:44.540850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:44.541795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:44.541795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:44.541795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:44.541795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:44.541795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:44.541795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:44.541795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:44.541795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:44.541795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:44.541795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:44.541795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:44.541795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:44.541795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 325/668] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:44.541795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc016a6-a179-4491-a615-cd272c23527d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:44.615457 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868054a6-c612-4b15-bf9c-d0dcd37db1d8 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:44.656375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f01a38-d21e-4a20-b6ef-82eedb2285c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:44.657061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f01a38-d21e-4a20-b6ef-82eedb2285c5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:44.657353 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f01a38-d21e-4a20-b6ef-82eedb2285c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:44.657636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f01a38-d21e-4a20-b6ef-82eedb2285c5 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:44.658477 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f01a38-d21e-4a20-b6ef-82eedb2285c5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:44.658794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f01a38-d21e-4a20-b6ef-82eedb2285c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:44.660418 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f01a38-d21e-4a20-b6ef-82eedb2285c5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:44.680956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 344/669] 10.222.0.69 () {56 vars in 1139 bytes} [Thu Jul 14 18:49:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:44.681717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f01a38-d21e-4a20-b6ef-82eedb2285c5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:44.694309 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc016a6-a179-4491-a615-cd272c23527d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:44.741171 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1678b34-4382-4578-aa97-b28d20bef73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:44.742087 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1678b34-4382-4578-aa97-b28d20bef73e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:44.742416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1678b34-4382-4578-aa97-b28d20bef73e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:44.743072 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1678b34-4382-4578-aa97-b28d20bef73e None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:44.743885 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1678b34-4382-4578-aa97-b28d20bef73e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:44.744198 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1678b34-4382-4578-aa97-b28d20bef73e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:44.746124 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1678b34-4382-4578-aa97-b28d20bef73e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:44.763194 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 326/670] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:44.764913 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1678b34-4382-4578-aa97-b28d20bef73e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:44.790049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:44.796910 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f01a38-d21e-4a20-b6ef-82eedb2285c5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:44.834365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:44.843730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870147e-b9aa-4b7c-994c-3af7223f330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:44.844706 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870147e-b9aa-4b7c-994c-3af7223f330d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:44.845136 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870147e-b9aa-4b7c-994c-3af7223f330d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:44.845585 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870147e-b9aa-4b7c-994c-3af7223f330d None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:44.846663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870147e-b9aa-4b7c-994c-3af7223f330d None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:44.847117 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870147e-b9aa-4b7c-994c-3af7223f330d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:44.849285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870147e-b9aa-4b7c-994c-3af7223f330d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:44.859921 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 345/671] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:49:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:44.860916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870147e-b9aa-4b7c-994c-3af7223f330d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:44.890151 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1678b34-4382-4578-aa97-b28d20bef73e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:44.934985 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd6b18-b4f2-4877-b9a5-0c21b053205f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:44.936026 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd6b18-b4f2-4877-b9a5-0c21b053205f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:44.936489 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd6b18-b4f2-4877-b9a5-0c21b053205f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:44.936923 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd6b18-b4f2-4877-b9a5-0c21b053205f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:44.977244 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870147e-b9aa-4b7c-994c-3af7223f330d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:45.020239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7be82-6a21-423a-bc17-7c78a01fc312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:45.021306 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7be82-6a21-423a-bc17-7c78a01fc312 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:45.021548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7be82-6a21-423a-bc17-7c78a01fc312 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:45.021922 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7be82-6a21-423a-bc17-7c78a01fc312 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:45.023999 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd6b18-b4f2-4877-b9a5-0c21b053205f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:45.025942 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd6b18-b4f2-4877-b9a5-0c21b053205f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:45.029943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7be82-6a21-423a-bc17-7c78a01fc312 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:45.031057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7be82-6a21-423a-bc17-7c78a01fc312 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:45.046316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-cfb7be82-6a21-423a-bc17-7c78a01fc312 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:45.046316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:45.046316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:45.046316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:45.046316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:45.046316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:45.046316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:45.046316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:45.046316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:45.046316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:45.046316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:45.046316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:45.046316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:45.046316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:45.046316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:45.047433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:45.047433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:45.047433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:45.047433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:45.047433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:45.047433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:45.047433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:45.047433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:45.047433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:45.047433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:45.047433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:45.047433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:45.047433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:45.047433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:45.047433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:45.047433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:45.047433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:45.048592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:45.048592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:45.048592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:45.048592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:45.048592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:45.048592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:45.048592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:45.048592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:45.050835 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 346/672] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:45.051826 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7be82-6a21-423a-bc17-7c78a01fc312 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:45.071468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:45.096855 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:45.146545 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 327/673] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:45.147731 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd6b18-b4f2-4877-b9a5-0c21b053205f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:45.177671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7be82-6a21-423a-bc17-7c78a01fc312 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:45.222682 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c10e1-51a9-4d93-bcf4-e91f916e9762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:45.223602 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c10e1-51a9-4d93-bcf4-e91f916e9762 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:45.223891 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c10e1-51a9-4d93-bcf4-e91f916e9762 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:45.224171 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c10e1-51a9-4d93-bcf4-e91f916e9762 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:45.236853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c10e1-51a9-4d93-bcf4-e91f916e9762 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:45.239768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c10e1-51a9-4d93-bcf4-e91f916e9762 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:45.261777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-707c10e1-51a9-4d93-bcf4-e91f916e9762 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:45.261777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:45.261777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:45.261777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:45.261777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:45.261777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:45.261777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:45.261777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:45.261777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:45.261777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:45.261777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:45.261777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:45.261777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:45.261777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:45.261777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:45.262717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:45.262717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:45.262717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:45.262717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:45.262717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:45.262717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:45.262717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:45.262717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:45.262717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:45.262717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:45.262717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:45.262717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:45.262717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:45.262717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:45.262717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:45.262717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:45.262717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:45.263616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:45.263616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:45.263616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:45.263616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:45.267439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 347/674] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:45.269178 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c10e1-51a9-4d93-bcf4-e91f916e9762 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:45.281277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd6b18-b4f2-4877-b9a5-0c21b053205f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:45.328304 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bacbbb-86ec-4b34-b88d-e2f6dbfb154c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:45.328970 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bacbbb-86ec-4b34-b88d-e2f6dbfb154c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:45.329256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bacbbb-86ec-4b34-b88d-e2f6dbfb154c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:45.329593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bacbbb-86ec-4b34-b88d-e2f6dbfb154c None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:45.330384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bacbbb-86ec-4b34-b88d-e2f6dbfb154c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:45.330742 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bacbbb-86ec-4b34-b88d-e2f6dbfb154c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:45.332241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bacbbb-86ec-4b34-b88d-e2f6dbfb154c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:45.347946 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:45.349505 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 328/675] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:45.350362 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bacbbb-86ec-4b34-b88d-e2f6dbfb154c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:45.367740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:45.398185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c10e1-51a9-4d93-bcf4-e91f916e9762 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:45.444257 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d167f-2663-4541-8f4b-7060ee7af4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:45.445641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d167f-2663-4541-8f4b-7060ee7af4dc None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:45.446372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d167f-2663-4541-8f4b-7060ee7af4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:45.447259 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d167f-2663-4541-8f4b-7060ee7af4dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:45.455860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bacbbb-86ec-4b34-b88d-e2f6dbfb154c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:45.461772 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d167f-2663-4541-8f4b-7060ee7af4dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:45.463471 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d167f-2663-4541-8f4b-7060ee7af4dc None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:45.488903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-bb4d167f-2663-4541-8f4b-7060ee7af4dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:45.488903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:45.488903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:45.488903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:45.488903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:45.488903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:45.488903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:45.488903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:45.488903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:45.488903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:45.488903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:45.488903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:45.488903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:45.488903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:45.488903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:45.489777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:45.489777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:45.489777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:45.489777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:45.489777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:45.489777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:45.489777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:45.489777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:45.489777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:45.489777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:45.489777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:45.489777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:45.489777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:45.489777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:45.489777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:45.489777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:45.489777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:45.490633 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:45.490633 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:45.490633 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:45.490633 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:45.492812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 348/676] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:45.493942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d167f-2663-4541-8f4b-7060ee7af4dc None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:45.494995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff5394-e6a0-421e-9fe5-c70c726d45b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:45.495852 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff5394-e6a0-421e-9fe5-c70c726d45b4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:45.496237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff5394-e6a0-421e-9fe5-c70c726d45b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:45.496913 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff5394-e6a0-421e-9fe5-c70c726d45b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:45.497627 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ff5394-e6a0-421e-9fe5-c70c726d45b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:45.498067 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ff5394-e6a0-421e-9fe5-c70c726d45b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:45.499563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ff5394-e6a0-421e-9fe5-c70c726d45b4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:45.515253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 329/677] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:45.516206 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff5394-e6a0-421e-9fe5-c70c726d45b4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:45.604160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:45.621006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d167f-2663-4541-8f4b-7060ee7af4dc None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:45.644153 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:45.645987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff5394-e6a0-421e-9fe5-c70c726d45b4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:45.664931 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6105266-bf33-438f-a17a-30b05e10f327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:45.666193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6105266-bf33-438f-a17a-30b05e10f327 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:45.666662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6105266-bf33-438f-a17a-30b05e10f327 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:45.667114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6105266-bf33-438f-a17a-30b05e10f327 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:45.668146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6105266-bf33-438f-a17a-30b05e10f327 None admin] RBAC: Adding query filter params (role.id=24846e82841b4b05b4d9e9d193a98868, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=e5d76eae62b54521a95f36284118fada) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:45.669114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6105266-bf33-438f-a17a-30b05e10f327 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:45.671070 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6105266-bf33-438f-a17a-30b05e10f327 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:45.680839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 349/678] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:49:45 2022] GET /identity/v3/role_assignments?user.id=e5d76eae62b54521a95f36284118fada&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=24846e82841b4b05b4d9e9d193a98868 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:45.681900 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6105266-bf33-438f-a17a-30b05e10f327 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:45.693366 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422756e6-5aca-4c05-beee-40c395ab0ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:45.693608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422756e6-5aca-4c05-beee-40c395ab0ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:45.694018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422756e6-5aca-4c05-beee-40c395ab0ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:45.694361 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422756e6-5aca-4c05-beee-40c395ab0ee8 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:45.695179 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422756e6-5aca-4c05-beee-40c395ab0ee8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:45.695579 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422756e6-5aca-4c05-beee-40c395ab0ee8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:45.697037 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422756e6-5aca-4c05-beee-40c395ab0ee8 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:45.721434 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 330/679] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:45.727365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422756e6-5aca-4c05-beee-40c395ab0ee8 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:45.804670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6105266-bf33-438f-a17a-30b05e10f327 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:45.846008 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a44a9-876f-475d-9496-2c6f45a1a0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:45.847219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a44a9-876f-475d-9496-2c6f45a1a0bf None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:45.848106 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a44a9-876f-475d-9496-2c6f45a1a0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:45.848568 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a44a9-876f-475d-9496-2c6f45a1a0bf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:45.858316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a44a9-876f-475d-9496-2c6f45a1a0bf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:45.860102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a44a9-876f-475d-9496-2c6f45a1a0bf None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:45.877639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-397a44a9-876f-475d-9496-2c6f45a1a0bf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 18:49:45.877639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:45.877639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:45.877639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:45.877639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:45.877639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:45.877639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:45.877639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:45.877639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:45.877639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:45.877639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:45.877639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:45.877639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:45.877639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:45.877639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:45.877639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:45.879235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:45.879235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:45.879235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:45.879235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:45.879235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:45.879235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:45.879235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:45.879235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:45.879235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:45.879235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:45.879235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:45.879235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:45.879235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:45.879235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:45.879235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:45.879235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:45.879235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:45.879235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:45.880158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:45.880158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:45.880158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:45.880158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:45.880158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:45.880158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:45.880158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:45.880158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:45.880158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 18:49:45.880158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:45.883469 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 350/680] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:49:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:45.891844 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a44a9-876f-475d-9496-2c6f45a1a0bf None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:45.928359 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422756e6-5aca-4c05-beee-40c395ab0ee8 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:45.954169 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:45.978335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200cc46-9ccf-41aa-8c97-b49f1252546c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:45.979523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200cc46-9ccf-41aa-8c97-b49f1252546c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:45.980533 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200cc46-9ccf-41aa-8c97-b49f1252546c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:45.981043 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200cc46-9ccf-41aa-8c97-b49f1252546c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:45.991074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200cc46-9ccf-41aa-8c97-b49f1252546c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:45.992868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200cc46-9ccf-41aa-8c97-b49f1252546c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:45.996166 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:46.012466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-f200cc46-9ccf-41aa-8c97-b49f1252546c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:46.012466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:46.012466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:46.012466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:46.012466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:46.012466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:46.012466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:46.012466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:46.012466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:46.012466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:46.012466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:46.012466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:46.012466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:46.012466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:46.012466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:46.022161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:46.022161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:46.022161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:46.022161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:46.022161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:46.022161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:46.022161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:46.022161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:46.022161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:46.022161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:46.022161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:46.022161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:46.022161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:46.022161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:46.022161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:46.022161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:46.022161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:46.023055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:46.023055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:46.023055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:46.023055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:46.023055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:46.023055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:46.023055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:46.023055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:46.023055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 331/681] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:46.023055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200cc46-9ccf-41aa-8c97-b49f1252546c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:46.026074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a44a9-876f-475d-9496-2c6f45a1a0bf None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:46.066384 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aceaef-f42a-4e1f-a9f1-07290e993979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:46.066990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aceaef-f42a-4e1f-a9f1-07290e993979 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:46.067329 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aceaef-f42a-4e1f-a9f1-07290e993979 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:46.067617 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aceaef-f42a-4e1f-a9f1-07290e993979 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:46.068538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aceaef-f42a-4e1f-a9f1-07290e993979 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:46.068855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aceaef-f42a-4e1f-a9f1-07290e993979 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:46.070369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aceaef-f42a-4e1f-a9f1-07290e993979 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:46.090188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 351/682] 10.222.0.69 () {56 vars in 1135 bytes} [Thu Jul 14 18:49:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:46.095931 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aceaef-f42a-4e1f-a9f1-07290e993979 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:46.143289 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200cc46-9ccf-41aa-8c97-b49f1252546c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:46.188976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41304649-1e92-4632-bfda-7afb69e65f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:46.189651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41304649-1e92-4632-bfda-7afb69e65f19 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:46.190401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41304649-1e92-4632-bfda-7afb69e65f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:46.190781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41304649-1e92-4632-bfda-7afb69e65f19 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:46.191619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41304649-1e92-4632-bfda-7afb69e65f19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:46.191933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41304649-1e92-4632-bfda-7afb69e65f19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:46.193767 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41304649-1e92-4632-bfda-7afb69e65f19 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:46.210658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 332/683] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:46.214181 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be52645-3d45-45e6-84ab-156b3c731357 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:46.214476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be52645-3d45-45e6-84ab-156b3c731357 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:46.214760 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be52645-3d45-45e6-84ab-156b3c731357 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:46.216627 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 333/684] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:46.217659 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be52645-3d45-45e6-84ab-156b3c731357 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:46.221139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aceaef-f42a-4e1f-a9f1-07290e993979 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:46.223727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:46.270883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a31cf0-cd6c-47a9-ac42-85a7a63ef37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:46.271593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a31cf0-cd6c-47a9-ac42-85a7a63ef37d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:46.271887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a31cf0-cd6c-47a9-ac42-85a7a63ef37d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:46.272170 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a31cf0-cd6c-47a9-ac42-85a7a63ef37d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:46.308873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:46.366714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be52645-3d45-45e6-84ab-156b3c731357 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:46.381615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a31cf0-cd6c-47a9-ac42-85a7a63ef37d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:46.383501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a31cf0-cd6c-47a9-ac42-85a7a63ef37d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:46.405714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0c84d-889b-4913-8a47-a45474f45ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:46.406383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0c84d-889b-4913-8a47-a45474f45ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:46.406720 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0c84d-889b-4913-8a47-a45474f45ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:46.407003 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0c84d-889b-4913-8a47-a45474f45ba3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:46.407869 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0c84d-889b-4913-8a47-a45474f45ba3 None admin] RBAC: Adding query filter params (role.id=24846e82841b4b05b4d9e9d193a98868, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=74b8c75b9cce442b8c430fab6a3fea17) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:46.408316 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0c84d-889b-4913-8a47-a45474f45ba3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:46.410489 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0c84d-889b-4913-8a47-a45474f45ba3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:46.423610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 334/685] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:49:46 2022] GET /identity/v3/role_assignments?user.id=74b8c75b9cce442b8c430fab6a3fea17&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=24846e82841b4b05b4d9e9d193a98868 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:46.425644 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad9c74-6c76-4c82-8988-494604c284f9 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:46.425996 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad9c74-6c76-4c82-8988-494604c284f9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:46.426282 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad9c74-6c76-4c82-8988-494604c284f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:46.459322 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-2fad9c74-6c76-4c82-8988-494604c284f9 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:46.460687 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2fad9c74-6c76-4c82-8988-494604c284f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:46.503577 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 352/686] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:46.513409 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a31cf0-cd6c-47a9-ac42-85a7a63ef37d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:46.519192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:46.567196 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 335/687] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:46.579201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad9c74-6c76-4c82-8988-494604c284f9 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:46.638468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:46.683951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a31cf0-cd6c-47a9-ac42-85a7a63ef37d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:46.695841 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad9c74-6c76-4c82-8988-494604c284f9 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:46.725867 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661e280-1b82-4e41-9a49-244ad24abf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:46.727014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661e280-1b82-4e41-9a49-244ad24abf6d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:46.728024 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661e280-1b82-4e41-9a49-244ad24abf6d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:46.728024 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661e280-1b82-4e41-9a49-244ad24abf6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:46.739749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661e280-1b82-4e41-9a49-244ad24abf6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:46.742860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194436ff-290f-4e89-b4bc-a523e1d94c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:46.742860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194436ff-290f-4e89-b4bc-a523e1d94c9d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:46.743083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661e280-1b82-4e41-9a49-244ad24abf6d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:46.743245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194436ff-290f-4e89-b4bc-a523e1d94c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:46.743245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194436ff-290f-4e89-b4bc-a523e1d94c9d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:46.743245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194436ff-290f-4e89-b4bc-a523e1d94c9d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:46.744211 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194436ff-290f-4e89-b4bc-a523e1d94c9d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:46.761726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-194436ff-290f-4e89-b4bc-a523e1d94c9d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 18:49:46.761726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:46.761726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:49:46.761726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:49:46.761726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:49:46.761726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:49:46.761726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:49:46.761726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:49:46.761726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 18:49:46.761726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:46.761726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:49:46.761726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:46.761726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:46.761726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:46.761726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:46.761726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:46.762836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:46.762836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:46.762836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:46.762836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:46.762836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:46.762836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:46.762836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:46.762836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:49:46.762836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:49:46.762836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:49:46.762836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:49:46.762836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:46.762836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:46.762836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:46.762836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:46.762836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:46.763893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:46.763893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:46.763893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:46.763893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:46.763893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:46.763893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:46.763893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:46.763893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:46.763893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:46.763893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:46.763893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:46.763893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:49:46.763893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:49:46.763893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 18:49:46.763893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:46.764747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-0661e280-1b82-4e41-9a49-244ad24abf6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:46.764747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:46.764747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:46.764747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:46.764747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:46.764747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:46.764747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:46.764747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:46.764747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:46.764747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:46.764747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:46.764747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:46.764747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:46.764747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:46.764747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:46.765555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:46.765555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:46.765555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:46.765555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:46.765555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:46.765555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:46.765555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:46.765555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:46.765555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:46.765555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:46.765555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:46.765555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:46.765555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:46.765555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:46.765555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:46.765555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:46.765555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:46.768213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:46.768213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:46.768213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:46.768213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:46.768213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 353/688] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:46.768213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9233fe-a030-4d00-9879-d8e0b1053345 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:46.769535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 336/689] 10.222.0.69 () {56 vars in 1129 bytes} [Thu Jul 14 18:49:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:46.769632 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9233fe-a030-4d00-9879-d8e0b1053345 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:46.769632 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9233fe-a030-4d00-9879-d8e0b1053345 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:46.769632 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 354/690] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:46.769978 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194436ff-290f-4e89-b4bc-a523e1d94c9d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:46.770845 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9233fe-a030-4d00-9879-d8e0b1053345 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:46.837981 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:46.885582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9233fe-a030-4d00-9879-d8e0b1053345 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:46.906413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194436ff-290f-4e89-b4bc-a523e1d94c9d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:46.920223 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:46.930839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ff599-affb-417b-bb94-29560fee91f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:46.932258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ff599-affb-417b-bb94-29560fee91f0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:46.932967 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ff599-affb-417b-bb94-29560fee91f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:46.933440 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ff599-affb-417b-bb94-29560fee91f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:46.934913 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ff599-affb-417b-bb94-29560fee91f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:46.935450 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ff599-affb-417b-bb94-29560fee91f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:46.937428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ff599-affb-417b-bb94-29560fee91f0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:46.943273 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fc841-6365-4a80-8d45-e577062d337b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:46.944027 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fc841-6365-4a80-8d45-e577062d337b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:46.944453 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fc841-6365-4a80-8d45-e577062d337b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:46.944808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fc841-6365-4a80-8d45-e577062d337b None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:46.945598 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464fc841-6365-4a80-8d45-e577062d337b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:46.945984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464fc841-6365-4a80-8d45-e577062d337b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:46.947519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464fc841-6365-4a80-8d45-e577062d337b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:46.957852 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 337/692] 10.222.0.69 () {56 vars in 1124 bytes} [Thu Jul 14 18:49:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:46.957852 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e14006-9d7c-4a99-af2f-33aef596838f None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:46.957852 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e14006-9d7c-4a99-af2f-33aef596838f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:46.957852 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e14006-9d7c-4a99-af2f-33aef596838f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:46.958353 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 355/692] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:46.959617 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ff599-affb-417b-bb94-29560fee91f0 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:46.974140 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a5e14006-9d7c-4a99-af2f-33aef596838f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:46.976100 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a5e14006-9d7c-4a99-af2f-33aef596838f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:47.064220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 338/693] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:47.065900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e14006-9d7c-4a99-af2f-33aef596838f None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:47.086535 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ff599-affb-417b-bb94-29560fee91f0 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:47.102696 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:47.125037 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483e73d-953e-4254-a624-6150c3336fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:47.125848 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483e73d-953e-4254-a624-6150c3336fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:47.126147 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483e73d-953e-4254-a624-6150c3336fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:47.126719 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483e73d-953e-4254-a624-6150c3336fe1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:47.134976 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6483e73d-953e-4254-a624-6150c3336fe1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:47.136209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6483e73d-953e-4254-a624-6150c3336fe1 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:47.153508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-6483e73d-953e-4254-a624-6150c3336fe1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:47.153508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:47.153508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:47.153508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:47.153508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:47.153508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:47.153508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:47.153508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:47.153508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:47.153508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:47.153508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:47.153508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:47.153508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:47.153508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:47.153508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:47.154428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:47.154428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:47.154428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:47.154428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:47.154428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:47.154428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:47.154428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:47.154428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:47.154428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:47.154428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:47.154428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:47.154428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:47.154428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:47.154428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:47.154428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:47.154428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:47.154428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:47.155333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:47.155333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:47.155333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:47.155333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:47.155333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:47.155333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:47.155333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:47.155333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:47.155333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 356/694] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:47.160107 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483e73d-953e-4254-a624-6150c3336fe1 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:47.172879 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e14006-9d7c-4a99-af2f-33aef596838f None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:47.181737 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:47.209093 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a245c86-7cc5-4f2f-8b70-9d6a08d52fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:47.210148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a245c86-7cc5-4f2f-8b70-9d6a08d52fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:47.210651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a245c86-7cc5-4f2f-8b70-9d6a08d52fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:47.211134 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a245c86-7cc5-4f2f-8b70-9d6a08d52fb1 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:47.212104 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a245c86-7cc5-4f2f-8b70-9d6a08d52fb1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:47.212640 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a245c86-7cc5-4f2f-8b70-9d6a08d52fb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:47.214198 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a245c86-7cc5-4f2f-8b70-9d6a08d52fb1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:47.220452 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 339/695] 10.222.0.69 () {56 vars in 1124 bytes} [Thu Jul 14 18:49:47 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:47.221730 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a245c86-7cc5-4f2f-8b70-9d6a08d52fb1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:47.278316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483e73d-953e-4254-a624-6150c3336fe1 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:47.337528 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a245c86-7cc5-4f2f-8b70-9d6a08d52fb1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:47.338162 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387d11e-c3d9-48bf-932f-c62d1e56ca5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:47.339051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387d11e-c3d9-48bf-932f-c62d1e56ca5e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:47.339380 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387d11e-c3d9-48bf-932f-c62d1e56ca5e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:47.339663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387d11e-c3d9-48bf-932f-c62d1e56ca5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:47.351818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5387d11e-c3d9-48bf-932f-c62d1e56ca5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:47.360513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-5387d11e-c3d9-48bf-932f-c62d1e56ca5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:49:47.360513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:47.360513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:47.360513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:47.360513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:47.360513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:47.360513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:47.360513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:47.360513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:47.360513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:47.360513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:47.360513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:47.360513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:47.360513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:47.360513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:47.360513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:47.361523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:47.361523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:47.361523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:47.361523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:47.361523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:47.361523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:47.361523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:47.361523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:47.361523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:47.361523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:47.361523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:47.361523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:47.361523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:47.361523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:47.361523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:47.361523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:47.362907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:47.362907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:47.362907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:47.362907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:47.362907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:47.362907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:47.362907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:47.362907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:47.362907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:47.362907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:47.362907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:49:47.362907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:47.362907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 357/696] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:49:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:47.367937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387d11e-c3d9-48bf-932f-c62d1e56ca5e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:47.391767 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e9115-4f9d-4041-9af5-ebad878e25ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:47.392842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e9115-4f9d-4041-9af5-ebad878e25ae None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:47.393347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e9115-4f9d-4041-9af5-ebad878e25ae None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:47.393862 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e9115-4f9d-4041-9af5-ebad878e25ae None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:47.394872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e9115-4f9d-4041-9af5-ebad878e25ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:47.395435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e9115-4f9d-4041-9af5-ebad878e25ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:47.397602 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e9115-4f9d-4041-9af5-ebad878e25ae None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:47.426433 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 340/697] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:47.432170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e9115-4f9d-4041-9af5-ebad878e25ae None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:47.441620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:47.465313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:47.527182 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387d11e-c3d9-48bf-932f-c62d1e56ca5e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:47.543464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e9115-4f9d-4041-9af5-ebad878e25ae None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:47.578159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82258590-fd5b-41dc-a579-92320565eee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:47.579030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82258590-fd5b-41dc-a579-92320565eee2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:47.584506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82258590-fd5b-41dc-a579-92320565eee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:47.584506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82258590-fd5b-41dc-a579-92320565eee2 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:47.584506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82258590-fd5b-41dc-a579-92320565eee2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:47.584506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82258590-fd5b-41dc-a579-92320565eee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:47.584506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82258590-fd5b-41dc-a579-92320565eee2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:47.599711 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 358/698] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:47.600749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd626c3-c495-4e17-abee-f0316d15cf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:47.601414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd626c3-c495-4e17-abee-f0316d15cf14 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:47.601698 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd626c3-c495-4e17-abee-f0316d15cf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:47.601972 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd626c3-c495-4e17-abee-f0316d15cf14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:47.602808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd626c3-c495-4e17-abee-f0316d15cf14 None admin] RBAC: Adding query filter params (role.id=24846e82841b4b05b4d9e9d193a98868, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=4368f68724f946eca53084a9ec7ff590) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:47.603127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd626c3-c495-4e17-abee-f0316d15cf14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:47.604594 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd626c3-c495-4e17-abee-f0316d15cf14 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:47.607125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82258590-fd5b-41dc-a579-92320565eee2 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:47.620325 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 341/699] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:49:47 2022] GET /identity/v3/role_assignments?user.id=4368f68724f946eca53084a9ec7ff590&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=24846e82841b4b05b4d9e9d193a98868 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:47.725166 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:47.732527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82258590-fd5b-41dc-a579-92320565eee2 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:47.777660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f96717-df1b-44fd-934e-02ad46961161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:47.778525 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f96717-df1b-44fd-934e-02ad46961161 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:47.778818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f96717-df1b-44fd-934e-02ad46961161 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:47.779098 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f96717-df1b-44fd-934e-02ad46961161 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:47.795647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f96717-df1b-44fd-934e-02ad46961161 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:47.797389 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f96717-df1b-44fd-934e-02ad46961161 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:47.819453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c6f96717-df1b-44fd-934e-02ad46961161 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:47.819453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:47.819453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:47.819453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:47.819453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:47.819453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:47.819453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:47.819453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:47.819453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:47.819453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:47.819453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:47.819453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:47.819453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:47.819453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:47.819453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:47.820396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:47.820396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:47.820396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:47.820396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:47.820396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:47.820396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:47.820396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:47.820396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:47.820396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:47.820396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:47.820396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:47.820396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:47.820396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:47.820396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:47.820396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:47.820396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:47.820396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:47.821257 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:47.821257 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:47.821257 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:47.821257 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:47.821257 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 359/700] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:47.826587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd626c3-c495-4e17-abee-f0316d15cf14 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:47.936840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:47.944051 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd626c3-c495-4e17-abee-f0316d15cf14 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:47.984097 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d7581-a645-43ef-92a2-b313ffaf42c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:47.984695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d7581-a645-43ef-92a2-b313ffaf42c1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:47.984985 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d7581-a645-43ef-92a2-b313ffaf42c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:47.985274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d7581-a645-43ef-92a2-b313ffaf42c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:47.986000 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d7581-a645-43ef-92a2-b313ffaf42c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:47.986314 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d7581-a645-43ef-92a2-b313ffaf42c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:47.987866 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d7581-a645-43ef-92a2-b313ffaf42c1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:48.002578 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 342/701] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:48.007694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f96717-df1b-44fd-934e-02ad46961161 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:48.111743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f96717-df1b-44fd-934e-02ad46961161 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:48.147410 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ddc45-eb7a-4988-9132-5557ba48f546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:48.148262 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ddc45-eb7a-4988-9132-5557ba48f546 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:48.148556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ddc45-eb7a-4988-9132-5557ba48f546 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:48.148838 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ddc45-eb7a-4988-9132-5557ba48f546 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:48.156691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ddc45-eb7a-4988-9132-5557ba48f546 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:48.158164 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ddc45-eb7a-4988-9132-5557ba48f546 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:48.173884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-e93ddc45-eb7a-4988-9132-5557ba48f546 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 18:49:48.173884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:48.173884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:48.173884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:48.173884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:48.173884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:48.173884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:48.173884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:48.173884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:48.173884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:48.173884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:48.173884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:48.173884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:48.173884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:48.173884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:48.173884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:48.174894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:48.174894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:48.174894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:48.174894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:48.174894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:48.174894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:48.174894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:48.174894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:48.174894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:48.174894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:48.174894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:48.174894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:48.174894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:48.174894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:48.174894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:48.174894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:48.174894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:48.174894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:48.175956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:48.175956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:48.175956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:48.175956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:48.175956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:48.175956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:48.175956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:48.175956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:48.175956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 18:49:48.175956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:48.175956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 360/702] 10.222.0.69 () {56 vars in 1093 bytes} [Thu Jul 14 18:49:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:48.181282 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d7581-a645-43ef-92a2-b313ffaf42c1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:48.289935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d7581-a645-43ef-92a2-b313ffaf42c1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:48.326068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009bf36-f57e-4e4c-9174-460451c6feff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:48.326750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009bf36-f57e-4e4c-9174-460451c6feff None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:48.327041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009bf36-f57e-4e4c-9174-460451c6feff None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:48.327370 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009bf36-f57e-4e4c-9174-460451c6feff None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:48.328219 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009bf36-f57e-4e4c-9174-460451c6feff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:48.328544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009bf36-f57e-4e4c-9174-460451c6feff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:48.330099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009bf36-f57e-4e4c-9174-460451c6feff None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:48.349300 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 343/703] 10.222.0.69 () {56 vars in 1133 bytes} [Thu Jul 14 18:49:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:48.354752 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ddc45-eb7a-4988-9132-5557ba48f546 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:48.366088 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:48.471374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ddc45-eb7a-4988-9132-5557ba48f546 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:48.479084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6838f-c923-475a-8a1b-440a00e3a293 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:48.479411 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6838f-c923-475a-8a1b-440a00e3a293 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:48.479757 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6838f-c923-475a-8a1b-440a00e3a293 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:48.481386 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 344/704] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:49:48.500368 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff48a28-308b-4b61-8f58-62c1e7e5e23b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:48.500705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff48a28-308b-4b61-8f58-62c1e7e5e23b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:48.500942 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff48a28-308b-4b61-8f58-62c1e7e5e23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:48.525358 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-eff48a28-308b-4b61-8f58-62c1e7e5e23b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:48.526819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-eff48a28-308b-4b61-8f58-62c1e7e5e23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:48.532137 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c94fd-3442-4497-86ef-523a0e7fa24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:48.532808 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c94fd-3442-4497-86ef-523a0e7fa24e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:48.533098 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c94fd-3442-4497-86ef-523a0e7fa24e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:48.533380 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c94fd-3442-4497-86ef-523a0e7fa24e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:48.544699 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:48.658160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c94fd-3442-4497-86ef-523a0e7fa24e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:48.660034 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c94fd-3442-4497-86ef-523a0e7fa24e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:48.664350 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 345/705] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:48.677659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:48.688466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff48a28-308b-4b61-8f58-62c1e7e5e23b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:48.824257 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff48a28-308b-4b61-8f58-62c1e7e5e23b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:48.848867 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 361/706] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:48.850865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa5c50-f287-4f6a-a47f-2b64a4f76e18 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:48.851392 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa5c50-f287-4f6a-a47f-2b64a4f76e18 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:48.851881 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa5c50-f287-4f6a-a47f-2b64a4f76e18 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:48.857515 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 362/707] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:48.858494 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa5c50-f287-4f6a-a47f-2b64a4f76e18 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:48.861753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ceea5-b7e8-4082-8f75-68ea8c93a080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:48.863102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ceea5-b7e8-4082-8f75-68ea8c93a080 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:48.863434 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ceea5-b7e8-4082-8f75-68ea8c93a080 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:48.863743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ceea5-b7e8-4082-8f75-68ea8c93a080 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:48.877777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ceea5-b7e8-4082-8f75-68ea8c93a080 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:48.880066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ceea5-b7e8-4082-8f75-68ea8c93a080 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:48.888748 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:48.901321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-8b6ceea5-b7e8-4082-8f75-68ea8c93a080 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:48.901321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:48.901321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:48.901321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:48.901321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:48.901321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:48.901321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:48.901321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:48.901321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:48.901321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:48.901321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:48.901321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:48.901321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:48.901321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:48.901321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:48.908689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:48.908689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:48.908689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:48.908689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:48.908689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:48.908689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:48.908689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:48.908689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:48.908689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:48.908689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:48.908689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:48.908689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:48.908689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:48.908689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:48.908689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:48.908689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:48.908689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:48.909730 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:48.909730 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:48.909730 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:48.909730 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:48.910406 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 346/708] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:49:48.912039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8ddda-48b0-44db-9f80-057aea3cb939 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:48.912521 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8ddda-48b0-44db-9f80-057aea3cb939 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:48.912982 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8ddda-48b0-44db-9f80-057aea3cb939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:48.931530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a6d8ddda-48b0-44db-9f80-057aea3cb939 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:48.933081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a6d8ddda-48b0-44db-9f80-057aea3cb939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:48.988145 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa5c50-f287-4f6a-a47f-2b64a4f76e18 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:49.012448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:49.013340 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 347/709] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:49.014413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8ddda-48b0-44db-9f80-057aea3cb939 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:49.057554 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8ec37-de77-4782-8bd7-11fb712ec3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:49.058414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8ec37-de77-4782-8bd7-11fb712ec3ca None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:49.058712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8ec37-de77-4782-8bd7-11fb712ec3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:49.059037 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8ec37-de77-4782-8bd7-11fb712ec3ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:49.072271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8ec37-de77-4782-8bd7-11fb712ec3ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:49.073855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8ec37-de77-4782-8bd7-11fb712ec3ca None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:49.092730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-0ab8ec37-de77-4782-8bd7-11fb712ec3ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:49.092730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:49.092730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:49.092730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:49.092730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:49.092730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:49.092730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:49.092730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:49.092730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:49.092730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:49.092730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:49.092730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:49.092730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:49.092730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:49.092730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:49.093681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:49.093681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:49.093681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:49.093681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:49.093681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:49.093681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:49.093681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:49.093681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:49.093681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:49.093681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:49.093681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:49.093681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:49.093681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:49.093681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:49.093681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:49.093681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:49.093681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:49.094982 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:49.094982 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:49.094982 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:49.094982 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:49.094982 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 363/710] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:49.095380 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8ec37-de77-4782-8bd7-11fb712ec3ca None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:49.142002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8ddda-48b0-44db-9f80-057aea3cb939 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:49.168999 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:49.181989 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570cf60-a84b-4a3a-8241-fc18f96513e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:49.182714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570cf60-a84b-4a3a-8241-fc18f96513e4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:49.183007 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570cf60-a84b-4a3a-8241-fc18f96513e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:49.183327 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570cf60-a84b-4a3a-8241-fc18f96513e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:49.184059 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570cf60-a84b-4a3a-8241-fc18f96513e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:49.184369 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570cf60-a84b-4a3a-8241-fc18f96513e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:49.185887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570cf60-a84b-4a3a-8241-fc18f96513e4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:49.203295 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 348/711] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:49.204337 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570cf60-a84b-4a3a-8241-fc18f96513e4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:49.222018 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8ec37-de77-4782-8bd7-11fb712ec3ca None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:49.267038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919177a-4c53-498f-86d3-d413ebd8926b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:49.268491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919177a-4c53-498f-86d3-d413ebd8926b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:49.269396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919177a-4c53-498f-86d3-d413ebd8926b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:49.269874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919177a-4c53-498f-86d3-d413ebd8926b None admin] PATH_INFO: `/v3/services/network` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:49.270741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919177a-4c53-498f-86d3-d413ebd8926b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:49.272333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919177a-4c53-498f-86d3-d413ebd8926b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:49.292973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-7919177a-4c53-498f-86d3-d413ebd8926b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 18:49:49.292973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:49.292973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:49:49.292973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:49:49.292973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:49:49.292973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:49:49.292973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:49:49.292973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:49:49.292973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 18:49:49.292973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:49.292973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:49:49.292973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:49.292973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:49.292973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:49.292973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:49.292973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:49.292973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:49.294067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:49.294067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:49.294067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:49.294067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:49.294067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:49.294067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:49.294067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:49:49.294067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:49:49.294067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:49:49.294067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:49:49.294067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:49.294067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:49.294067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:49.294067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:49.294067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:49.294067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:49.294988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:49.294988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:49.294988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:49.294988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:49.294988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:49.294988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:49.294988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:49.294988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:49.294988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:49.294988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:49.294988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:49:49.294988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:49:49.294988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 18:49:49.294988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:49.297164 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 364/712] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:49 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:49.297314 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919177a-4c53-498f-86d3-d413ebd8926b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:49.332934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:49.341451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570cf60-a84b-4a3a-8241-fc18f96513e4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:49.384249 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40d103-ec38-40f8-b900-43ca38b20175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:49.384849 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40d103-ec38-40f8-b900-43ca38b20175 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:49.385136 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40d103-ec38-40f8-b900-43ca38b20175 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:49.385418 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40d103-ec38-40f8-b900-43ca38b20175 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:49.386153 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40d103-ec38-40f8-b900-43ca38b20175 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:49.386469 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40d103-ec38-40f8-b900-43ca38b20175 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:49.388005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40d103-ec38-40f8-b900-43ca38b20175 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:49.407580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 349/713] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:49.408676 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40d103-ec38-40f8-b900-43ca38b20175 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:49.422785 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919177a-4c53-498f-86d3-d413ebd8926b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:49.461876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:49.469510 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35196d9-1b95-4b42-aaf0-e5b4858583f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:49.470393 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35196d9-1b95-4b42-aaf0-e5b4858583f5 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:49.471306 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35196d9-1b95-4b42-aaf0-e5b4858583f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:49.472312 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35196d9-1b95-4b42-aaf0-e5b4858583f5 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:49.473478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35196d9-1b95-4b42-aaf0-e5b4858583f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:49.475643 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35196d9-1b95-4b42-aaf0-e5b4858583f5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:49.522536 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40d103-ec38-40f8-b900-43ca38b20175 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:49.559473 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 365/714] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 14 18:49:49 2022] POST /identity/v3/users => generated 254 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:49:49.561173 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35196d9-1b95-4b42-aaf0-e5b4858583f5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:49.568324 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d1325-02ef-4a8c-b13e-d2b829edf1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:49.570465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d1325-02ef-4a8c-b13e-d2b829edf1ff None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:49.570465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d1325-02ef-4a8c-b13e-d2b829edf1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:49.572786 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d1325-02ef-4a8c-b13e-d2b829edf1ff None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:49.573739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d1325-02ef-4a8c-b13e-d2b829edf1ff None admin] RBAC: Adding query filter params (name=network) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:49.574070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d1325-02ef-4a8c-b13e-d2b829edf1ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:49.577819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d1325-02ef-4a8c-b13e-d2b829edf1ff None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:49.590613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 350/715] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 14 18:49:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:49.598474 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d1325-02ef-4a8c-b13e-d2b829edf1ff None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:49.746003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:49.766113 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35196d9-1b95-4b42-aaf0-e5b4858583f5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:49.777372 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d1325-02ef-4a8c-b13e-d2b829edf1ff None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:49.788306 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:49.810546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c48fb5-44d8-448d-baf0-e6781de2b862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:49.812515 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c48fb5-44d8-448d-baf0-e6781de2b862 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:49.812515 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c48fb5-44d8-448d-baf0-e6781de2b862 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:49.812515 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c48fb5-44d8-448d-baf0-e6781de2b862 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:49.822911 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f6792-21fe-4b8d-accc-86eeeb38d7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:49.823628 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f6792-21fe-4b8d-accc-86eeeb38d7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:49.823918 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f6792-21fe-4b8d-accc-86eeeb38d7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:49.824196 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f6792-21fe-4b8d-accc-86eeeb38d7a6 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:49.824916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f6792-21fe-4b8d-accc-86eeeb38d7a6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:49.825227 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f6792-21fe-4b8d-accc-86eeeb38d7a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:49.826676 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f6792-21fe-4b8d-accc-86eeeb38d7a6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:49.828980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c48fb5-44d8-448d-baf0-e6781de2b862 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:49.830443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c48fb5-44d8-448d-baf0-e6781de2b862 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:49.832493 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 351/716] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 14 18:49:49 2022] GET /identity/v3/services?type=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:49.834135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3cb87-a6a8-4af0-bd3f-835d7f81358b None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:49.834135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3cb87-a6a8-4af0-bd3f-835d7f81358b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:49.834304 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3cb87-a6a8-4af0-bd3f-835d7f81358b None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:49.835528 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 352/717] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:49.841227 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3853f9-00f3-49f9-96bc-85e6986c90a8 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:49.841546 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3853f9-00f3-49f9-96bc-85e6986c90a8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:49.841833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3853f9-00f3-49f9-96bc-85e6986c90a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:49.846160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c4c48fb5-44d8-448d-baf0-e6781de2b862 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:49.846160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:49.846160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:49.846160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:49.846160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:49.846160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:49.846160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:49.846160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:49.846160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:49.846160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:49.846160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:49.846160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:49.846160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:49.846160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:49.846160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:49.847756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:49.847756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:49.847756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:49.847756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:49.847756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:49.847756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:49.847756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:49.847756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:49.847756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:49.847756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:49.847756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:49.847756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:49.847756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:49.847756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:49.847756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:49.847756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:49.847756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:49.851146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:49.851146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:49.851146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:49.851146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:49.851146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:49.851146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:49.851146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:49.851146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:49.851146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 366/718] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:49.856824 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c48fb5-44d8-448d-baf0-e6781de2b862 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:49.864245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3d3853f9-00f3-49f9-96bc-85e6986c90a8 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:49.865614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3d3853f9-00f3-49f9-96bc-85e6986c90a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:50.005196 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c48fb5-44d8-448d-baf0-e6781de2b862 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:50.009043 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 353/719] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:49:50.032813 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3853f9-00f3-49f9-96bc-85e6986c90a8 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:50.048503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b7bb0-8cdf-4fca-a255-2a9348de3050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:50.049283 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b7bb0-8cdf-4fca-a255-2a9348de3050 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:50.050282 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b7bb0-8cdf-4fca-a255-2a9348de3050 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:50.050603 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b7bb0-8cdf-4fca-a255-2a9348de3050 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:50.051497 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b7bb0-8cdf-4fca-a255-2a9348de3050 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:50.052058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b7bb0-8cdf-4fca-a255-2a9348de3050 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:50.053418 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b7bb0-8cdf-4fca-a255-2a9348de3050 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:50.067507 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:50.071355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 367/720] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:50.084176 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b7bb0-8cdf-4fca-a255-2a9348de3050 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:50.146953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3853f9-00f3-49f9-96bc-85e6986c90a8 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:50.148860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:50.184148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179063c3-6213-4ddd-8486-adf5bba17908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:50.185228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179063c3-6213-4ddd-8486-adf5bba17908 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:50.185746 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179063c3-6213-4ddd-8486-adf5bba17908 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:50.186229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179063c3-6213-4ddd-8486-adf5bba17908 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:50.187194 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179063c3-6213-4ddd-8486-adf5bba17908 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:50.188778 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179063c3-6213-4ddd-8486-adf5bba17908 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:50.267452 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b7bb0-8cdf-4fca-a255-2a9348de3050 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:50.307760 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 354/721] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 14 18:49:50 2022] POST /identity/v3/services => generated 251 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:49:50.315072 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabb7e5-f9df-40ba-ac5f-f8790a8a51ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:50.315882 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabb7e5-f9df-40ba-ac5f-f8790a8a51ad None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:50.316168 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabb7e5-f9df-40ba-ac5f-f8790a8a51ad None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:50.316441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabb7e5-f9df-40ba-ac5f-f8790a8a51ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:50.317354 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feabb7e5-f9df-40ba-ac5f-f8790a8a51ad None admin] RBAC: Adding query filter params (role.id=27fc14631c494ca0b2dc7f708be016c2, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=8d27d6df5cb64697adebd961dfbec324) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:50.317673 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feabb7e5-f9df-40ba-ac5f-f8790a8a51ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:50.319200 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feabb7e5-f9df-40ba-ac5f-f8790a8a51ad None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:50.326639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 368/722] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:49:50 2022] GET /identity/v3/role_assignments?user.id=8d27d6df5cb64697adebd961dfbec324&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:50.417989 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbe67d-8bc6-4813-a2a9-deed33fb8585 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:50.418285 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbe67d-8bc6-4813-a2a9-deed33fb8585 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:50.418579 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbe67d-8bc6-4813-a2a9-deed33fb8585 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:50.420268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 355/723] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:50.426066 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c7085-595c-4020-bac3-96d6f0cfeed5 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:50.426357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c7085-595c-4020-bac3-96d6f0cfeed5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:50.426644 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c7085-595c-4020-bac3-96d6f0cfeed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:50.445905 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-610c7085-595c-4020-bac3-96d6f0cfeed5 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:50.447212 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-610c7085-595c-4020-bac3-96d6f0cfeed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:50.529304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:50.543059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 369/724] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:50.566578 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbe67d-8bc6-4813-a2a9-deed33fb8585 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:50.684600 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbe67d-8bc6-4813-a2a9-deed33fb8585 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:50.722485 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51fe3e-3fd0-4beb-9eeb-e937ecd215a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:50.724715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51fe3e-3fd0-4beb-9eeb-e937ecd215a2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:50.725015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51fe3e-3fd0-4beb-9eeb-e937ecd215a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:50.725309 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51fe3e-3fd0-4beb-9eeb-e937ecd215a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:50.736416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a51fe3e-3fd0-4beb-9eeb-e937ecd215a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:50.744894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-5a51fe3e-3fd0-4beb-9eeb-e937ecd215a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:49:50.744894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:50.744894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:50.744894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:50.744894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:50.744894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:50.744894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:50.744894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:50.744894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:50.744894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:50.744894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:50.744894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:50.744894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:50.744894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:50.744894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:50.744894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:50.745808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:50.745808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:50.745808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:50.745808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:50.745808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:50.745808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:50.745808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:50.745808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:50.745808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:50.745808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:50.745808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:50.745808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:50.745808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:50.745808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:50.745808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:50.745808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:50.746624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:50.746624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:50.746624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:50.746624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:50.746624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:50.746624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:50.746624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:50.746624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:50.746624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:50.746624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:50.746624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:49:50.746624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:50.746624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 356/725] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:49:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:50.752397 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610c7085-595c-4020-bac3-96d6f0cfeed5 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:50.872685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610c7085-595c-4020-bac3-96d6f0cfeed5 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:50.914727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51225dfc-65be-43df-86f6-07d8083f3c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:50.915455 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51225dfc-65be-43df-86f6-07d8083f3c8a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:50.915740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51225dfc-65be-43df-86f6-07d8083f3c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:50.916018 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51225dfc-65be-43df-86f6-07d8083f3c8a None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:50.916881 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51225dfc-65be-43df-86f6-07d8083f3c8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:50.917195 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51225dfc-65be-43df-86f6-07d8083f3c8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:50.918578 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51225dfc-65be-43df-86f6-07d8083f3c8a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:50.929369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 370/726] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:50.934840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51fe3e-3fd0-4beb-9eeb-e937ecd215a2 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:50.978921 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:51.092040 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51fe3e-3fd0-4beb-9eeb-e937ecd215a2 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:51.171938 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbc7e7-0870-4a95-9c85-d5058edf3dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:51.172661 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbc7e7-0870-4a95-9c85-d5058edf3dfb None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:51.172949 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbc7e7-0870-4a95-9c85-d5058edf3dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:51.173227 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbc7e7-0870-4a95-9c85-d5058edf3dfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:51.185480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbc7e7-0870-4a95-9c85-d5058edf3dfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:51.186920 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbc7e7-0870-4a95-9c85-d5058edf3dfb None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:51.213779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-5efbc7e7-0870-4a95-9c85-d5058edf3dfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:51.213779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:51.213779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:51.213779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:51.213779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:51.213779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:51.213779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:51.213779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:51.213779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:51.213779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:51.213779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:51.213779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:51.213779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:51.213779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:51.213779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:51.214694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:51.214694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:51.214694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:51.214694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:51.214694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:51.214694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:51.214694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:51.214694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:51.214694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:51.214694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:51.214694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:51.214694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:51.214694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:51.214694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:51.214694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:51.214694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:51.214694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:51.215626 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:51.215626 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:51.215626 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:51.215626 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:51.217311 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 357/727] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:51.222289 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51225dfc-65be-43df-86f6-07d8083f3c8a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:51.326616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:51.381185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51225dfc-65be-43df-86f6-07d8083f3c8a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:51.434396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c24b6b-9d75-4732-8d16-6502087da9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:51.434997 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c24b6b-9d75-4732-8d16-6502087da9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:51.435312 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c24b6b-9d75-4732-8d16-6502087da9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:51.435596 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c24b6b-9d75-4732-8d16-6502087da9d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:51.436326 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c24b6b-9d75-4732-8d16-6502087da9d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:51.436643 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c24b6b-9d75-4732-8d16-6502087da9d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:51.438035 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c24b6b-9d75-4732-8d16-6502087da9d6 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:51.450865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 371/728] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:51.456353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbc7e7-0870-4a95-9c85-d5058edf3dfb None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:51.771441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:51.806992 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbc7e7-0870-4a95-9c85-d5058edf3dfb None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:51.855919 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faec247-bb9b-47f6-8dcc-5b9b3ead9e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:51.856740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faec247-bb9b-47f6-8dcc-5b9b3ead9e48 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:51.857027 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faec247-bb9b-47f6-8dcc-5b9b3ead9e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:51.857303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faec247-bb9b-47f6-8dcc-5b9b3ead9e48 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:51.865425 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faec247-bb9b-47f6-8dcc-5b9b3ead9e48 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:51.866918 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faec247-bb9b-47f6-8dcc-5b9b3ead9e48 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:51.887039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-0faec247-bb9b-47f6-8dcc-5b9b3ead9e48 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 18:49:51.887039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:51.887039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:51.887039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:51.887039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:51.887039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:51.887039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:51.887039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:51.887039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:51.887039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:51.887039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:51.887039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:51.887039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:51.887039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:51.887039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:51.887039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:51.888060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:51.888060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:51.888060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:51.888060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:51.888060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:51.888060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:51.888060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:51.888060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:51.888060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:51.888060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:51.888060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:51.888060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:51.888060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:51.888060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:51.888060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:51.888060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:51.888060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:51.888060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:51.888984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:51.888984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:51.888984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:51.888984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:51.888984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:51.888984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:51.888984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:51.888984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:51.888984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 18:49:51.888984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:51.890686 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 358/729] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:49:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:51.896045 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c24b6b-9d75-4732-8d16-6502087da9d6 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:52.030586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1a3ec-ed29-4925-8e0b-68477c46754d None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:52.031078 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1a3ec-ed29-4925-8e0b-68477c46754d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:52.031078 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1a3ec-ed29-4925-8e0b-68477c46754d None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:52.032737 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 359/730] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:52.038281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad4360-65e0-42e3-a6a3-1af07376753d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:52.038695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad4360-65e0-42e3-a6a3-1af07376753d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:52.038980 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad4360-65e0-42e3-a6a3-1af07376753d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:52.051468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c24b6b-9d75-4732-8d16-6502087da9d6 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:52.064979 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c7ad4360-65e0-42e3-a6a3-1af07376753d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:52.066796 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c7ad4360-65e0-42e3-a6a3-1af07376753d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:52.070694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:52.114544 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a135187-746f-464c-bb10-f1767d8b3388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:52.115259 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a135187-746f-464c-bb10-f1767d8b3388 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:52.115566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a135187-746f-464c-bb10-f1767d8b3388 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:52.115844 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a135187-746f-464c-bb10-f1767d8b3388 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:52.116703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a135187-746f-464c-bb10-f1767d8b3388 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:52.117019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a135187-746f-464c-bb10-f1767d8b3388 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:52.118483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a135187-746f-464c-bb10-f1767d8b3388 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:52.162046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 372/731] 10.222.0.69 () {56 vars in 1135 bytes} [Thu Jul 14 18:49:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:52.166079 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a135187-746f-464c-bb10-f1767d8b3388 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:52.213655 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 360/732] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:52 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 18:49:52.245913 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad4360-65e0-42e3-a6a3-1af07376753d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:52.318183 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:52.354866 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad4360-65e0-42e3-a6a3-1af07376753d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:52.363933 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a135187-746f-464c-bb10-f1767d8b3388 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:52.391436 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8212a-009c-49af-b65b-fa5506e4d526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:52.392249 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8212a-009c-49af-b65b-fa5506e4d526 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:52.392537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8212a-009c-49af-b65b-fa5506e4d526 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:52.392818 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8212a-009c-49af-b65b-fa5506e4d526 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:52.405537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8212a-009c-49af-b65b-fa5506e4d526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:52.413896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-31a8212a-009c-49af-b65b-fa5506e4d526 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:52.413896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:52.413896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:52.413896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:52.413896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:52.413896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:52.413896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:52.413896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:52.413896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:52.413896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:52.413896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:52.413896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:52.413896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:52.413896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:52.413896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:52.413896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:52.414854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:52.414854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:52.414854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:52.414854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:52.414854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:52.414854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:52.414854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:52.414854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:52.414854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:52.414854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:52.414854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:52.414854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:52.414854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:52.414854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:52.414854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:52.414854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:52.415745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:52.415745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:52.415745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:52.415745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:52.415745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:52.415745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:52.415745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:52.415745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:52.415745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:52.415745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:52.415745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:52.415745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:52.415745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 361/733] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:52.422565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8212a-009c-49af-b65b-fa5506e4d526 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:52.443239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829574f4-1849-42a3-beda-8257dc3466d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:52.444208 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829574f4-1849-42a3-beda-8257dc3466d0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:52.444692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829574f4-1849-42a3-beda-8257dc3466d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:52.445195 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829574f4-1849-42a3-beda-8257dc3466d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:52.517604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:52.534938 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8212a-009c-49af-b65b-fa5506e4d526 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:52.548203 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829574f4-1849-42a3-beda-8257dc3466d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:52.550610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829574f4-1849-42a3-beda-8257dc3466d0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:52.563556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:52.573441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189eb16d-3a24-4d4c-9128-170ef217c1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:52.574234 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189eb16d-3a24-4d4c-9128-170ef217c1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:52.574535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189eb16d-3a24-4d4c-9128-170ef217c1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:52.574823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189eb16d-3a24-4d4c-9128-170ef217c1c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:52.575621 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189eb16d-3a24-4d4c-9128-170ef217c1c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:52.575941 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189eb16d-3a24-4d4c-9128-170ef217c1c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:52.577600 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189eb16d-3a24-4d4c-9128-170ef217c1c6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:52.587196 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 362/734] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:49:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:52.588694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115112e-3d66-4d5e-910f-63c9563a83ed None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:52.588994 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115112e-3d66-4d5e-910f-63c9563a83ed None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:52.589283 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115112e-3d66-4d5e-910f-63c9563a83ed None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:52.590757 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 363/735] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:52.592617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115112e-3d66-4d5e-910f-63c9563a83ed None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:52.658014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 373/736] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:52 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 18:49:52.659577 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829f8e1-8b4f-4302-b89d-126aabf9bb4e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:52.659987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829f8e1-8b4f-4302-b89d-126aabf9bb4e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:52.660363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829f8e1-8b4f-4302-b89d-126aabf9bb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:52.677319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d829f8e1-8b4f-4302-b89d-126aabf9bb4e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:52.678669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d829f8e1-8b4f-4302-b89d-126aabf9bb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:52.719802 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115112e-3d66-4d5e-910f-63c9563a83ed None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:52.763899 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd68169-86ef-4b6b-9acf-9a34033b6d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:52.763899 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd68169-86ef-4b6b-9acf-9a34033b6d68 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:52.764218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd68169-86ef-4b6b-9acf-9a34033b6d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:52.764466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd68169-86ef-4b6b-9acf-9a34033b6d68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:52.769613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:52.776311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 374/737] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:52 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 18:49:52.777853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829f8e1-8b4f-4302-b89d-126aabf9bb4e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:52.788524 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd68169-86ef-4b6b-9acf-9a34033b6d68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:52.790019 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd68169-86ef-4b6b-9acf-9a34033b6d68 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:52.821883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-bbd68169-86ef-4b6b-9acf-9a34033b6d68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:52.821883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:52.821883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:52.821883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:52.821883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:52.821883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:52.821883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:52.821883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:52.821883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:52.821883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:52.821883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:52.821883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:52.821883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:52.821883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:52.821883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:52.822857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:52.822857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:52.822857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:52.822857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:52.822857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:52.822857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:52.822857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:52.822857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:52.822857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:52.822857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:52.822857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:52.822857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:52.822857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:52.822857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:52.822857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:52.822857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:52.822857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:52.823820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:52.823820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:52.823820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:52.823820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:52.827289 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 364/738] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:52.828275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd68169-86ef-4b6b-9acf-9a34033b6d68 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:52.870331 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:52.894584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829f8e1-8b4f-4302-b89d-126aabf9bb4e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:52.931668 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd68169-86ef-4b6b-9acf-9a34033b6d68 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:52.933835 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30facfef-e912-42ec-b53f-78cd7c77d89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:52.934645 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30facfef-e912-42ec-b53f-78cd7c77d89e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:52.934933 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30facfef-e912-42ec-b53f-78cd7c77d89e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:52.935249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30facfef-e912-42ec-b53f-78cd7c77d89e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:52.945442 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30facfef-e912-42ec-b53f-78cd7c77d89e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:52.946923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30facfef-e912-42ec-b53f-78cd7c77d89e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:52.964327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-30facfef-e912-42ec-b53f-78cd7c77d89e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:52.964327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:52.964327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:52.964327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:52.964327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:52.964327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:52.964327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:52.964327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:52.964327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:52.964327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:52.964327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:52.964327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:52.964327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:52.964327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:52.964327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:52.965381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:52.965381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:52.965381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:52.965381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:52.965381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:52.965381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:52.965381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:52.965381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:52.965381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:52.965381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:52.965381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:52.965381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:52.965381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:52.965381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:52.965381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:52.965381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:52.965381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:52.966292 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:52.966292 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:52.966292 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:52.966292 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:52.966292 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 375/739] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:52.966292 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30facfef-e912-42ec-b53f-78cd7c77d89e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:52.968371 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd9745-3822-4251-bcc6-a38797e061be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:52.969039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd9745-3822-4251-bcc6-a38797e061be None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:52.969328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd9745-3822-4251-bcc6-a38797e061be None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:52.969604 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd9745-3822-4251-bcc6-a38797e061be None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:52.970630 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd9745-3822-4251-bcc6-a38797e061be None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:52.972591 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd9745-3822-4251-bcc6-a38797e061be None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:53.019239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:53.059987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 365/740] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 14 18:49:52 2022] POST /identity/v3/services => generated 240 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:49:53.061211 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306719bf-6704-43cf-ad4f-b8a1667263dc None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:53.061502 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306719bf-6704-43cf-ad4f-b8a1667263dc None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:53.061775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306719bf-6704-43cf-ad4f-b8a1667263dc None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:53.063188 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 366/741] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:53.066779 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30facfef-e912-42ec-b53f-78cd7c77d89e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:53.068792 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809ab26-0278-42bc-b9fe-6b890a0b2f05 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:53.069087 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809ab26-0278-42bc-b9fe-6b890a0b2f05 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:53.069368 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809ab26-0278-42bc-b9fe-6b890a0b2f05 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:53.082428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 367/742] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:53.082428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809ab26-0278-42bc-b9fe-6b890a0b2f05 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:53.118286 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9c388-bbbd-484e-a2b8-96bf2566d2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:53.118973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9c388-bbbd-484e-a2b8-96bf2566d2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:53.119290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9c388-bbbd-484e-a2b8-96bf2566d2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:53.119571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9c388-bbbd-484e-a2b8-96bf2566d2a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:53.120313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9c388-bbbd-484e-a2b8-96bf2566d2a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:53.120624 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9c388-bbbd-484e-a2b8-96bf2566d2a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:53.122069 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9c388-bbbd-484e-a2b8-96bf2566d2a8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:53.135895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 376/743] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:53.137248 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594f159-cbc3-4196-9735-aca2bc4f4b9c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:53.137543 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594f159-cbc3-4196-9735-aca2bc4f4b9c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:53.137825 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594f159-cbc3-4196-9735-aca2bc4f4b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:53.157848 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9594f159-cbc3-4196-9735-aca2bc4f4b9c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:53.159603 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9594f159-cbc3-4196-9735-aca2bc4f4b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:53.214652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:53.220063 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809ab26-0278-42bc-b9fe-6b890a0b2f05 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:53.235611 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 377/744] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:49:53.237030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef07190-e74f-4efa-b3cf-2a7b6d090ec1 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:53.237414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef07190-e74f-4efa-b3cf-2a7b6d090ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:53.237704 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef07190-e74f-4efa-b3cf-2a7b6d090ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:53.254076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-bef07190-e74f-4efa-b3cf-2a7b6d090ec1 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:53.255312 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bef07190-e74f-4efa-b3cf-2a7b6d090ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:53.258489 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c16e3-dbcc-4294-82f2-9c90252f28a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:53.259392 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c16e3-dbcc-4294-82f2-9c90252f28a3 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:53.259625 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c16e3-dbcc-4294-82f2-9c90252f28a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:53.259879 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c16e3-dbcc-4294-82f2-9c90252f28a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:53.260607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c16e3-dbcc-4294-82f2-9c90252f28a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:53.260917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c16e3-dbcc-4294-82f2-9c90252f28a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:53.262282 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c16e3-dbcc-4294-82f2-9c90252f28a3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:53.276147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 368/745] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:53.277666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c16e3-dbcc-4294-82f2-9c90252f28a3 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:53.292927 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:53.331585 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 378/746] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:49:53.332321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef07190-e74f-4efa-b3cf-2a7b6d090ec1 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:53.410861 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c16e3-dbcc-4294-82f2-9c90252f28a3 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:53.444385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef07190-e74f-4efa-b3cf-2a7b6d090ec1 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:53.447666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4129d77-7405-4a9b-866c-02393815c342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:53.448490 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4129d77-7405-4a9b-866c-02393815c342 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:53.448777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4129d77-7405-4a9b-866c-02393815c342 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:53.449054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4129d77-7405-4a9b-866c-02393815c342 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:53.457192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4129d77-7405-4a9b-866c-02393815c342 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:53.458640 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4129d77-7405-4a9b-866c-02393815c342 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:53.474237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-e4129d77-7405-4a9b-866c-02393815c342 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 18:49:53.474237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:53.474237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:53.474237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:53.474237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:53.474237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:53.474237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:53.474237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:53.474237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:53.474237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:53.474237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:53.474237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:53.474237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:53.474237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:53.474237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:53.474237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:53.475262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:53.475262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:53.475262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:53.475262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:53.475262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:53.475262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:53.475262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:53.475262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:53.475262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:53.475262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:53.475262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:53.475262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:53.475262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:53.475262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:53.475262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:53.475262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:53.475262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:53.475262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:53.476224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:53.476224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:53.476224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:53.476224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:53.476224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:53.476224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:53.476224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:53.476224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:53.476224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 18:49:53.476224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:53.476224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 369/747] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 14 18:49:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:53.476224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4129d77-7405-4a9b-866c-02393815c342 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:53.487086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f25bc-61ce-4483-bf5a-52283ae4b6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:53.487919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f25bc-61ce-4483-bf5a-52283ae4b6ad None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:53.488488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f25bc-61ce-4483-bf5a-52283ae4b6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:53.488631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f25bc-61ce-4483-bf5a-52283ae4b6ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:53.497870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:53.498851 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f25bc-61ce-4483-bf5a-52283ae4b6ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:53.506556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-a15f25bc-61ce-4483-bf5a-52283ae4b6ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:49:53.506556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:53.506556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:53.506556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:53.506556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:53.506556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:53.506556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:53.506556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:53.506556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:53.506556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:53.506556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:53.506556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:53.506556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:53.506556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:53.506556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:53.506556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:53.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:53.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:53.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:53.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:53.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:53.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:53.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:53.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:53.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:53.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:53.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:53.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:53.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:53.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:53.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:53.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:53.514149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:53.514149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:53.514149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:53.514149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:53.514149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:53.514149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:53.514149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:53.514149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:53.514149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:53.514149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:53.514149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:49:53.514149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:53.514149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 379/748] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:49:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:53.514149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f25bc-61ce-4483-bf5a-52283ae4b6ad None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:53.541320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:53.593715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4129d77-7405-4a9b-866c-02393815c342 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:53.620601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f25bc-61ce-4483-bf5a-52283ae4b6ad None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:53.628128 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97993bd8-99c9-4be2-bd29-e8c7a583e5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:53.628929 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97993bd8-99c9-4be2-bd29-e8c7a583e5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:53.629214 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97993bd8-99c9-4be2-bd29-e8c7a583e5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:53.629488 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97993bd8-99c9-4be2-bd29-e8c7a583e5b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:53.636479 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97993bd8-99c9-4be2-bd29-e8c7a583e5b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:53.637916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97993bd8-99c9-4be2-bd29-e8c7a583e5b8 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:53.654146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-97993bd8-99c9-4be2-bd29-e8c7a583e5b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:53.654146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:53.654146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:53.654146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:53.654146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:53.654146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:53.654146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:53.654146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:53.654146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:53.654146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:53.654146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:53.654146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:53.654146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:53.654146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:53.654146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:53.655055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:53.655055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:53.655055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:53.655055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:53.655055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:53.655055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:53.655055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:53.655055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:53.655055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:53.655055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:53.655055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:53.655055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:53.655055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:53.655055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:53.655055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:53.655055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:53.655055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:53.656079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:53.656079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:53.656079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:53.656079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:53.656079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:53.656079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:53.656079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:53.656079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:53.656079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 370/749] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:53.656555 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97993bd8-99c9-4be2-bd29-e8c7a583e5b8 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:53.662057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b8c92-0ac3-4766-9384-441547cf5034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:53.662827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b8c92-0ac3-4766-9384-441547cf5034 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:53.663872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b8c92-0ac3-4766-9384-441547cf5034 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:53.663872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b8c92-0ac3-4766-9384-441547cf5034 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:53.664111 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b8c92-0ac3-4766-9384-441547cf5034 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:53.665491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b8c92-0ac3-4766-9384-441547cf5034 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:53.678475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-126b8c92-0ac3-4766-9384-441547cf5034 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 18:49:53.678475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:53.678475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:49:53.678475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:49:53.678475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:49:53.678475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:49:53.678475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:49:53.678475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:49:53.678475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 18:49:53.678475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:53.678475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:49:53.678475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:53.678475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:53.678475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:53.678475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:53.678475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:53.679631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:53.679631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:53.679631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:53.679631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:53.679631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:53.679631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:53.679631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:53.679631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:49:53.679631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:49:53.679631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:49:53.679631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:49:53.679631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:53.679631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:53.679631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:53.679631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:53.679631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:53.680503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:53.680503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:53.680503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:53.680503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:53.680503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:53.680503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:53.680503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:53.680503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:53.680503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:53.680503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:53.680503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:53.680503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:49:53.680503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:49:53.680503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 18:49:53.680503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:53.680503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 380/750] 10.222.0.69 () {56 vars in 1129 bytes} [Thu Jul 14 18:49:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:53.681331 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b8c92-0ac3-4766-9384-441547cf5034 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:53.748037 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:53.773836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97993bd8-99c9-4be2-bd29-e8c7a583e5b8 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:53.778361 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:53.788451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b8c92-0ac3-4766-9384-441547cf5034 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:53.810427 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830c3d6-5863-4f49-a4a2-8254343eb3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:53.811107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830c3d6-5863-4f49-a4a2-8254343eb3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:53.811431 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830c3d6-5863-4f49-a4a2-8254343eb3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:53.811713 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830c3d6-5863-4f49-a4a2-8254343eb3b6 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:53.812559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830c3d6-5863-4f49-a4a2-8254343eb3b6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:53.812872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830c3d6-5863-4f49-a4a2-8254343eb3b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:53.814328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830c3d6-5863-4f49-a4a2-8254343eb3b6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:53.824012 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c881a3a-c9c8-44c9-af0e-425d0dd7e710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:53.824719 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c881a3a-c9c8-44c9-af0e-425d0dd7e710 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:53.825007 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c881a3a-c9c8-44c9-af0e-425d0dd7e710 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:53.825291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c881a3a-c9c8-44c9-af0e-425d0dd7e710 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:53.826038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c881a3a-c9c8-44c9-af0e-425d0dd7e710 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:53.826352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c881a3a-c9c8-44c9-af0e-425d0dd7e710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:53.827762 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c881a3a-c9c8-44c9-af0e-425d0dd7e710 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:53.837613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 371/751] 10.222.0.69 () {56 vars in 1149 bytes} [Thu Jul 14 18:49:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:53.837613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830c3d6-5863-4f49-a4a2-8254343eb3b6 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:53.841755 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 381/752] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:53.846340 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c881a3a-c9c8-44c9-af0e-425d0dd7e710 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:53.938727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830c3d6-5863-4f49-a4a2-8254343eb3b6 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:53.953912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c881a3a-c9c8-44c9-af0e-425d0dd7e710 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:53.973219 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07524f43-956a-4eb9-adf2-fdf0dd341167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:53.973880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07524f43-956a-4eb9-adf2-fdf0dd341167 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:53.974174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07524f43-956a-4eb9-adf2-fdf0dd341167 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:53.974457 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07524f43-956a-4eb9-adf2-fdf0dd341167 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:53.975267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07524f43-956a-4eb9-adf2-fdf0dd341167 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:53.975589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07524f43-956a-4eb9-adf2-fdf0dd341167 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:53.977027 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07524f43-956a-4eb9-adf2-fdf0dd341167 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:53.979557 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:53.994598 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 372/753] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:53.994765 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa2230-b537-4d54-b4fb-5c6ee6b8c636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:53.994884 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07524f43-956a-4eb9-adf2-fdf0dd341167 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:53.994961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa2230-b537-4d54-b4fb-5c6ee6b8c636 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:53.994961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa2230-b537-4d54-b4fb-5c6ee6b8c636 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:53.994961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa2230-b537-4d54-b4fb-5c6ee6b8c636 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:53.994961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa2230-b537-4d54-b4fb-5c6ee6b8c636 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:53.994961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa2230-b537-4d54-b4fb-5c6ee6b8c636 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:53.996949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa2230-b537-4d54-b4fb-5c6ee6b8c636 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:54.005488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 382/754] 10.222.0.69 () {56 vars in 1124 bytes} [Thu Jul 14 18:49:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:54.006314 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa2230-b537-4d54-b4fb-5c6ee6b8c636 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:54.026197 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:54.109795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa2230-b537-4d54-b4fb-5c6ee6b8c636 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:54.116267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07524f43-956a-4eb9-adf2-fdf0dd341167 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:54.146936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabaed25-17f1-4e81-bb03-b85c3079e5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:54.147782 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabaed25-17f1-4e81-bb03-b85c3079e5ac None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:54.148075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabaed25-17f1-4e81-bb03-b85c3079e5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:54.148363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabaed25-17f1-4e81-bb03-b85c3079e5ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:54.152354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2d8d0-4242-4ec2-aeb3-f49ae23a4f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:54.153000 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2d8d0-4242-4ec2-aeb3-f49ae23a4f7a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:54.153287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2d8d0-4242-4ec2-aeb3-f49ae23a4f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:54.153563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2d8d0-4242-4ec2-aeb3-f49ae23a4f7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:54.159752 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabaed25-17f1-4e81-bb03-b85c3079e5ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:54.161227 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabaed25-17f1-4e81-bb03-b85c3079e5ac None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:54.179385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-eabaed25-17f1-4e81-bb03-b85c3079e5ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:54.179385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:54.179385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:54.179385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:54.179385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:54.179385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:54.179385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:54.179385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:54.179385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:54.179385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:54.179385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:54.179385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:54.179385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:54.179385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:54.179385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:54.180316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:54.180316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:54.180316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:54.180316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:54.180316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:54.180316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:54.180316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:54.180316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:54.180316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:54.180316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:54.180316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:54.180316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:54.180316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:54.180316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:54.180316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:54.180316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:54.180316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:54.181249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:54.181249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:54.181249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:54.181249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:54.181249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 383/755] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:54.181521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabaed25-17f1-4e81-bb03-b85c3079e5ac None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:54.226389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:54.233884 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2d8d0-4242-4ec2-aeb3-f49ae23a4f7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:54.235396 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2d8d0-4242-4ec2-aeb3-f49ae23a4f7a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:54.256936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:54.288124 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabaed25-17f1-4e81-bb03-b85c3079e5ac None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:54.326467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b05adc-7273-48d7-8d9d-1f5d7577145a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:54.327127 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b05adc-7273-48d7-8d9d-1f5d7577145a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:54.327444 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b05adc-7273-48d7-8d9d-1f5d7577145a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:54.327725 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b05adc-7273-48d7-8d9d-1f5d7577145a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:54.328555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b05adc-7273-48d7-8d9d-1f5d7577145a None admin] RBAC: Adding query filter params (role.id=27fc14631c494ca0b2dc7f708be016c2, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=4368f68724f946eca53084a9ec7ff590) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:54.328870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b05adc-7273-48d7-8d9d-1f5d7577145a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:54.330323 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b05adc-7273-48d7-8d9d-1f5d7577145a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:54.339319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 384/756] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:49:54 2022] GET /identity/v3/role_assignments?user.id=4368f68724f946eca53084a9ec7ff590&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:54.341154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b05adc-7273-48d7-8d9d-1f5d7577145a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:54.346898 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 373/757] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:49:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:49:54.347464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2d8d0-4242-4ec2-aeb3-f49ae23a4f7a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:54.475225 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:54.492676 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2d8d0-4242-4ec2-aeb3-f49ae23a4f7a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:54.550672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55707f8-ad97-420f-91f3-e3e952568541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:54.551736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55707f8-ad97-420f-91f3-e3e952568541 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:54.552223 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55707f8-ad97-420f-91f3-e3e952568541 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:54.552795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55707f8-ad97-420f-91f3-e3e952568541 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:54.554296 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55707f8-ad97-420f-91f3-e3e952568541 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:54.554819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55707f8-ad97-420f-91f3-e3e952568541 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:54.556871 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55707f8-ad97-420f-91f3-e3e952568541 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:54.574105 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 374/758] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:54.576025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55707f8-ad97-420f-91f3-e3e952568541 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:54.581676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b05adc-7273-48d7-8d9d-1f5d7577145a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:54.628716 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad24bce-c18b-4704-9dd9-0548f6355526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:54.629396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad24bce-c18b-4704-9dd9-0548f6355526 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:54.629693 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad24bce-c18b-4704-9dd9-0548f6355526 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:54.630212 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad24bce-c18b-4704-9dd9-0548f6355526 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:54.630931 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad24bce-c18b-4704-9dd9-0548f6355526 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:54.631266 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad24bce-c18b-4704-9dd9-0548f6355526 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:54.633099 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad24bce-c18b-4704-9dd9-0548f6355526 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:54.634832 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:54.646541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 385/759] 10.222.0.69 () {56 vars in 1124 bytes} [Thu Jul 14 18:49:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:54.646541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad24bce-c18b-4704-9dd9-0548f6355526 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:54.701866 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55707f8-ad97-420f-91f3-e3e952568541 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:54.746651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b3664-8e89-4db0-b4c4-3c4f80b6024a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:54.748286 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b3664-8e89-4db0-b4c4-3c4f80b6024a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:54.748573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b3664-8e89-4db0-b4c4-3c4f80b6024a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:54.748850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b3664-8e89-4db0-b4c4-3c4f80b6024a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:54.763568 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b3664-8e89-4db0-b4c4-3c4f80b6024a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:54.765654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b3664-8e89-4db0-b4c4-3c4f80b6024a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:54.769652 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad24bce-c18b-4704-9dd9-0548f6355526 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:54.789840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-ac5b3664-8e89-4db0-b4c4-3c4f80b6024a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:54.789840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:54.789840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:54.789840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:54.789840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:54.789840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:54.789840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:54.789840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:54.789840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:54.789840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:54.789840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:54.789840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:54.789840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:54.789840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:54.789840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:54.790809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:54.790809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:54.790809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:54.790809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:54.790809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:54.790809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:54.790809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:54.790809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:54.790809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:54.790809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:54.790809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:54.790809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:54.790809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:54.790809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:54.790809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:54.790809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:54.790809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:54.791783 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:54.791783 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:54.791783 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:54.791783 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:54.791783 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 375/760] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:54.795199 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b3664-8e89-4db0-b4c4-3c4f80b6024a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:54.806547 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:54.826952 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ca736-aa82-407c-b680-deb46a9dc26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:54.828843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ca736-aa82-407c-b680-deb46a9dc26d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:54.829346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ca736-aa82-407c-b680-deb46a9dc26d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:54.829957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ca736-aa82-407c-b680-deb46a9dc26d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:54.840916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ca736-aa82-407c-b680-deb46a9dc26d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:54.843149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ca736-aa82-407c-b680-deb46a9dc26d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:54.859664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-d65ca736-aa82-407c-b680-deb46a9dc26d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 18:49:54.859664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:54.859664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:54.859664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:54.859664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:54.859664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:54.859664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:54.859664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:54.859664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:54.859664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:54.859664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:54.859664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:54.859664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:54.859664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:54.859664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:54.859664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:54.860670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:54.860670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:54.860670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:54.860670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:54.860670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:54.860670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:54.860670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:54.860670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:54.860670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:54.860670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:54.860670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:54.860670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:54.860670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:54.860670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:54.860670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:54.860670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:54.860670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:54.860670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:54.861613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:54.861613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:54.861613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:54.861613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:54.861613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:54.861613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:54.861613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:54.861613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:54.861613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 18:49:54.861613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:54.868611 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 386/761] 10.222.0.69 () {56 vars in 1093 bytes} [Thu Jul 14 18:49:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:54.869626 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ca736-aa82-407c-b680-deb46a9dc26d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:54.912698 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b3664-8e89-4db0-b4c4-3c4f80b6024a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:54.940264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:54.950044 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f5ca9-db24-4a02-bb22-d959a34721a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:54.950780 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f5ca9-db24-4a02-bb22-d959a34721a5 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:54.951070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f5ca9-db24-4a02-bb22-d959a34721a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:54.951384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f5ca9-db24-4a02-bb22-d959a34721a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:54.954801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f5ca9-db24-4a02-bb22-d959a34721a5 None admin] RBAC: Adding query filter params (interface=public, service_id=46045c7ebf024d0a89338b81b62113a9, region_id=RegionOne) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:54.954801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f5ca9-db24-4a02-bb22-d959a34721a5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:54.954801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f5ca9-db24-4a02-bb22-d959a34721a5 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:54.971135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 376/762] 10.222.0.69 () {56 vars in 1251 bytes} [Thu Jul 14 18:49:54 2022] GET /identity/v3/endpoints?service_id=46045c7ebf024d0a89338b81b62113a9&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:54.972871 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f5ca9-db24-4a02-bb22-d959a34721a5 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:54.990820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ca736-aa82-407c-b680-deb46a9dc26d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:55.028106 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a267a-89d8-4e6c-8bc6-19e975fe1e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:55.028879 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a267a-89d8-4e6c-8bc6-19e975fe1e7e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:55.029169 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a267a-89d8-4e6c-8bc6-19e975fe1e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:55.029443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a267a-89d8-4e6c-8bc6-19e975fe1e7e None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:55.030262 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a267a-89d8-4e6c-8bc6-19e975fe1e7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:55.030572 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a267a-89d8-4e6c-8bc6-19e975fe1e7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:55.032030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a267a-89d8-4e6c-8bc6-19e975fe1e7e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:55.045797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 387/763] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:55.046886 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a267a-89d8-4e6c-8bc6-19e975fe1e7e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:55.055663 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:55.088783 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f5ca9-db24-4a02-bb22-d959a34721a5 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:55.125464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed205b-2f6b-4b6b-8f09-740e53d6aa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:55.126147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed205b-2f6b-4b6b-8f09-740e53d6aa8a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:55.126439 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed205b-2f6b-4b6b-8f09-740e53d6aa8a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:55.126716 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed205b-2f6b-4b6b-8f09-740e53d6aa8a None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:55.128138 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed205b-2f6b-4b6b-8f09-740e53d6aa8a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:55.128504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed205b-2f6b-4b6b-8f09-740e53d6aa8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:55.130062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed205b-2f6b-4b6b-8f09-740e53d6aa8a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:55.151504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 377/764] 10.222.0.69 () {56 vars in 1133 bytes} [Thu Jul 14 18:49:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:55.152738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed205b-2f6b-4b6b-8f09-740e53d6aa8a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:55.157779 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a267a-89d8-4e6c-8bc6-19e975fe1e7e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:55.176052 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:55.193821 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62de7f0-88b8-4e6c-aa34-9ab7131fc8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:55.194485 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62de7f0-88b8-4e6c-aa34-9ab7131fc8e9 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:55.196233 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62de7f0-88b8-4e6c-aa34-9ab7131fc8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:55.196233 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62de7f0-88b8-4e6c-aa34-9ab7131fc8e9 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:55.196233 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62de7f0-88b8-4e6c-aa34-9ab7131fc8e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:55.197357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62de7f0-88b8-4e6c-aa34-9ab7131fc8e9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:55.203463 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 388/765] 10.222.0.69 () {56 vars in 1087 bytes} [Thu Jul 14 18:49:55 2022] GET /identity/v3/services => generated 772 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:55.203825 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62de7f0-88b8-4e6c-aa34-9ab7131fc8e9 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:55.276432 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed205b-2f6b-4b6b-8f09-740e53d6aa8a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:55.317165 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:55.328185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013f875-4c6c-40a8-be81-82dcaf5f8bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:55.329018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013f875-4c6c-40a8-be81-82dcaf5f8bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:55.329301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013f875-4c6c-40a8-be81-82dcaf5f8bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:55.329573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013f875-4c6c-40a8-be81-82dcaf5f8bb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:55.332873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62de7f0-88b8-4e6c-aa34-9ab7131fc8e9 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:55.340300 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013f875-4c6c-40a8-be81-82dcaf5f8bb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:55.342793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013f875-4c6c-40a8-be81-82dcaf5f8bb6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:55.362567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-5013f875-4c6c-40a8-be81-82dcaf5f8bb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:55.362567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:55.362567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:55.362567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:55.362567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:55.362567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:55.362567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:55.362567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:55.362567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:55.362567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:55.362567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:55.362567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:55.362567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:55.362567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:55.362567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:55.363517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:55.363517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:55.363517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:55.363517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:55.363517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:55.363517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:55.363517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:55.363517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:55.363517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:55.363517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:55.363517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:55.363517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:55.363517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:55.363517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:55.363517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:55.363517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:55.363517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:55.364395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:55.364395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:55.364395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:55.364395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:55.364395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:55.364395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:55.364395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:55.364395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:55.364395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 378/766] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:55.365937 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d42e8-192e-45aa-9353-a2a51fc3a889 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:55.365937 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d42e8-192e-45aa-9353-a2a51fc3a889 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:55.366990 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d42e8-192e-45aa-9353-a2a51fc3a889 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:55.368496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 379/767] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:55.375240 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d42e8-192e-45aa-9353-a2a51fc3a889 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:55.375400 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6464e74-571a-4edc-a6c5-3e082f347f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:55.375400 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6464e74-571a-4edc-a6c5-3e082f347f08 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:55.375400 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6464e74-571a-4edc-a6c5-3e082f347f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:55.375400 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6464e74-571a-4edc-a6c5-3e082f347f08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:55.392887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6464e74-571a-4edc-a6c5-3e082f347f08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:55.394889 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6464e74-571a-4edc-a6c5-3e082f347f08 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:55.412459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c6464e74-571a-4edc-a6c5-3e082f347f08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:55.412459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:55.412459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:55.412459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:55.412459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:55.412459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:55.412459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:55.412459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:55.412459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:55.412459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:55.412459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:55.412459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:55.412459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:55.412459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:55.412459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:55.413443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:55.413443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:55.413443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:55.413443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:55.413443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:55.413443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:55.413443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:55.413443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:55.413443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:55.413443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:55.413443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:55.413443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:55.413443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:55.413443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:55.413443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:55.413443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:55.413443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:55.414316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:55.414316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:55.414316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:55.414316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:55.414316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 389/768] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:55.415140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b259243-cd8f-4ab2-99c2-ac9276ef0a20 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:55.415459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b259243-cd8f-4ab2-99c2-ac9276ef0a20 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:55.415741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b259243-cd8f-4ab2-99c2-ac9276ef0a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:55.433428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0b259243-cd8f-4ab2-99c2-ac9276ef0a20 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:55.442153 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0b259243-cd8f-4ab2-99c2-ac9276ef0a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:55.464029 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:55.497538 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d42e8-192e-45aa-9353-a2a51fc3a889 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:55.506275 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 390/769] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:49:55.507498 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b259243-cd8f-4ab2-99c2-ac9276ef0a20 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:55.540070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfce7f-d9d9-4e8d-8184-2ef156140cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:55.541236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfce7f-d9d9-4e8d-8184-2ef156140cfa None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:55.541763 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfce7f-d9d9-4e8d-8184-2ef156140cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:55.542251 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfce7f-d9d9-4e8d-8184-2ef156140cfa None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:55.543346 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfce7f-d9d9-4e8d-8184-2ef156140cfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:55.543857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfce7f-d9d9-4e8d-8184-2ef156140cfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:55.545689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfce7f-d9d9-4e8d-8184-2ef156140cfa None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:55.560106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 380/770] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:55.561308 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfce7f-d9d9-4e8d-8184-2ef156140cfa None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:55.611685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b259243-cd8f-4ab2-99c2-ac9276ef0a20 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:55.617549 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:55.648140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856691ca-243c-4104-ad54-65fdb7f174c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:55.648822 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856691ca-243c-4104-ad54-65fdb7f174c5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:55.649280 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856691ca-243c-4104-ad54-65fdb7f174c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:55.649565 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856691ca-243c-4104-ad54-65fdb7f174c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:55.650285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856691ca-243c-4104-ad54-65fdb7f174c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:55.650595 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856691ca-243c-4104-ad54-65fdb7f174c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:55.652280 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856691ca-243c-4104-ad54-65fdb7f174c5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:55.665266 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 391/771] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:55.666277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856691ca-243c-4104-ad54-65fdb7f174c5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:55.676135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfce7f-d9d9-4e8d-8184-2ef156140cfa None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:55.692792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:55.717030 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ab583-cea5-488e-a32e-526d37256fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:55.718223 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ab583-cea5-488e-a32e-526d37256fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:55.718737 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ab583-cea5-488e-a32e-526d37256fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:55.719238 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ab583-cea5-488e-a32e-526d37256fc2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:55.720089 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ab583-cea5-488e-a32e-526d37256fc2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:55.721782 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ab583-cea5-488e-a32e-526d37256fc2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:55.735596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-c03ab583-cea5-488e-a32e-526d37256fc2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 18:49:55.735596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:55.735596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:49:55.735596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:49:55.735596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:49:55.735596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:49:55.735596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:49:55.735596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:49:55.735596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 18:49:55.735596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:55.735596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:49:55.735596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:55.735596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:55.735596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:55.735596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:55.735596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:55.735596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:55.736596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:55.736596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:55.736596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:55.736596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:55.736596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:55.736596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:55.736596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:49:55.736596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:49:55.736596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:49:55.736596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:49:55.736596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:55.736596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:55.736596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:55.736596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:55.736596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:55.736596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:55.738024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:55.738024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:55.738024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:55.738024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:55.738024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:55.738024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:55.738024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:55.738024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:55.738024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:55.738024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:55.738024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:49:55.738024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:49:55.738024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 18:49:55.738024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:55.740464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 381/772] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:55 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:55.741391 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ab583-cea5-488e-a32e-526d37256fc2 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:55.771558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856691ca-243c-4104-ad54-65fdb7f174c5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:55.811910 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7126e8-ea93-4455-97b0-2b18e35354fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:55.812672 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7126e8-ea93-4455-97b0-2b18e35354fd None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:55.813286 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7126e8-ea93-4455-97b0-2b18e35354fd None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:55.813666 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7126e8-ea93-4455-97b0-2b18e35354fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:55.815228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7126e8-ea93-4455-97b0-2b18e35354fd None admin] RBAC: Adding query filter params (role.id=24846e82841b4b05b4d9e9d193a98868, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=4ef0fad9a71d43bfbaba9c87dfed41c4) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:55.815653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7126e8-ea93-4455-97b0-2b18e35354fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:55.817766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7126e8-ea93-4455-97b0-2b18e35354fd None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:55.829015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 392/773] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:49:55 2022] GET /identity/v3/role_assignments?user.id=4ef0fad9a71d43bfbaba9c87dfed41c4&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=24846e82841b4b05b4d9e9d193a98868 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:55.830344 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7126e8-ea93-4455-97b0-2b18e35354fd None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:55.855610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ab583-cea5-488e-a32e-526d37256fc2 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:55.904857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:55.960721 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135cede-87fa-4826-8943-47e663fd2e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:55.961564 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135cede-87fa-4826-8943-47e663fd2e2e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:55.961853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135cede-87fa-4826-8943-47e663fd2e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:55.962135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135cede-87fa-4826-8943-47e663fd2e2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:55.979279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135cede-87fa-4826-8943-47e663fd2e2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:55.980788 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135cede-87fa-4826-8943-47e663fd2e2e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:55.991567 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:56.004940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-9135cede-87fa-4826-8943-47e663fd2e2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:56.004940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:56.004940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:56.004940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:56.004940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:56.004940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:56.004940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:56.004940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:56.004940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:56.004940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:56.004940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:56.004940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:56.004940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:56.004940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:56.004940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:56.014583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:56.014583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:56.014583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:56.014583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:56.014583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:56.014583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:56.014583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:56.014583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:56.014583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:56.014583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:56.014583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:56.014583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:56.014583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:56.014583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:56.014583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:56.014583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:56.014583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:56.016005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:56.016005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:56.016005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:56.016005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:56.016005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:56.016005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:56.016005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:56.016005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:56.016005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 382/774] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:56.016005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135cede-87fa-4826-8943-47e663fd2e2e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:56.016558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7126e8-ea93-4455-97b0-2b18e35354fd None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:56.068410 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3c37b-5f72-4936-9653-bd2fb04af0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:56.069060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3c37b-5f72-4936-9653-bd2fb04af0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:56.069355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3c37b-5f72-4936-9653-bd2fb04af0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:56.069634 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3c37b-5f72-4936-9653-bd2fb04af0e0 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:56.070357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3c37b-5f72-4936-9653-bd2fb04af0e0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:56.070669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3c37b-5f72-4936-9653-bd2fb04af0e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:56.073373 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3c37b-5f72-4936-9653-bd2fb04af0e0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:56.085006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 393/775] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 14 18:49:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:56.088594 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3c37b-5f72-4936-9653-bd2fb04af0e0 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:56.162435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135cede-87fa-4826-8943-47e663fd2e2e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:56.204265 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28e894-6b82-4355-9717-3b0a4f32a13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:56.204932 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28e894-6b82-4355-9717-3b0a4f32a13e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:56.205216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28e894-6b82-4355-9717-3b0a4f32a13e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:56.205494 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28e894-6b82-4355-9717-3b0a4f32a13e None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:56.206831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28e894-6b82-4355-9717-3b0a4f32a13e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:56.207141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28e894-6b82-4355-9717-3b0a4f32a13e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:56.214657 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28e894-6b82-4355-9717-3b0a4f32a13e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:56.240150 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 383/776] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:56.246054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28e894-6b82-4355-9717-3b0a4f32a13e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:56.281804 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3c37b-5f72-4936-9653-bd2fb04af0e0 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:56.288296 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:56.345760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a8ad5-2583-4577-b7d4-7d4aa74ec971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:56.347081 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a8ad5-2583-4577-b7d4-7d4aa74ec971 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:56.347398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a8ad5-2583-4577-b7d4-7d4aa74ec971 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:56.347669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a8ad5-2583-4577-b7d4-7d4aa74ec971 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:56.348383 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a8ad5-2583-4577-b7d4-7d4aa74ec971 None admin] RBAC: Adding query filter params (type=network) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:56.348695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a8ad5-2583-4577-b7d4-7d4aa74ec971 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:56.350078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a8ad5-2583-4577-b7d4-7d4aa74ec971 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:56.363582 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28e894-6b82-4355-9717-3b0a4f32a13e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:56.365647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 394/777] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 14 18:49:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:56.372955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a8ad5-2583-4577-b7d4-7d4aa74ec971 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:56.407138 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bb22f-8c07-4493-bba8-9864a8b6bafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:56.408503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bb22f-8c07-4493-bba8-9864a8b6bafe None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:56.409672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bb22f-8c07-4493-bba8-9864a8b6bafe None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:56.410193 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bb22f-8c07-4493-bba8-9864a8b6bafe None admin] PATH_INFO: `/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/8d27d6df5cb64697adebd961dfbec324/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:56.412510 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255bb22f-8c07-4493-bba8-9864a8b6bafe None admin] RBAC: Authorizing `identity:create_grant(project_id=6003dba8ddd247e7842d28ea7ca5238c, user_id=8d27d6df5cb64697adebd961dfbec324, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:56.414059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:56.414310 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255bb22f-8c07-4493-bba8-9864a8b6bafe None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:56.506870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 384/778] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:49:56 2022] PUT /identity/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/8d27d6df5cb64697adebd961dfbec324/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:49:56.508346 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735db3c-e9e4-4d95-9005-3d106ad1006d None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:56.508645 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735db3c-e9e4-4d95-9005-3d106ad1006d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:56.508927 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735db3c-e9e4-4d95-9005-3d106ad1006d None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:56.510301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 385/779] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:56.524116 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a8ad5-2583-4577-b7d4-7d4aa74ec971 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:56.532624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a246b-5669-40a7-ac44-46191e5bbe8d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:56.532624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a246b-5669-40a7-ac44-46191e5bbe8d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:56.532624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a246b-5669-40a7-ac44-46191e5bbe8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:56.559215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-1c2a246b-5669-40a7-ac44-46191e5bbe8d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:56.560768 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1c2a246b-5669-40a7-ac44-46191e5bbe8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:56.610141 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc61ab2-c6c3-4ffa-88fb-1f74dcb1bd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:56.610754 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc61ab2-c6c3-4ffa-88fb-1f74dcb1bd26 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:56.611042 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc61ab2-c6c3-4ffa-88fb-1f74dcb1bd26 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:56.611346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc61ab2-c6c3-4ffa-88fb-1f74dcb1bd26 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:56.612293 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc61ab2-c6c3-4ffa-88fb-1f74dcb1bd26 None admin] RBAC: Adding query filter params (interface=public, service_id=036c0e9e242340f79c75b6e4d8a3f693, region_id=RegionOne) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:56.612612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc61ab2-c6c3-4ffa-88fb-1f74dcb1bd26 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:56.614049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc61ab2-c6c3-4ffa-88fb-1f74dcb1bd26 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:56.632102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 395/780] 10.222.0.69 () {56 vars in 1251 bytes} [Thu Jul 14 18:49:56 2022] GET /identity/v3/endpoints?service_id=036c0e9e242340f79c75b6e4d8a3f693&interface=public®ion_id=RegionOne => generated 194 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:56.639307 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc61ab2-c6c3-4ffa-88fb-1f74dcb1bd26 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:56.681096 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 386/781] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:49:56.725433 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a246b-5669-40a7-ac44-46191e5bbe8d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:56.757872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:56.788749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc61ab2-c6c3-4ffa-88fb-1f74dcb1bd26 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:56.811617 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:56.836923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c10f5d-ed24-48c7-8043-0b0a67371599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:56.837600 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c10f5d-ed24-48c7-8043-0b0a67371599 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:56.837887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c10f5d-ed24-48c7-8043-0b0a67371599 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:56.838165 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c10f5d-ed24-48c7-8043-0b0a67371599 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:56.838848 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c10f5d-ed24-48c7-8043-0b0a67371599 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:56.840321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c10f5d-ed24-48c7-8043-0b0a67371599 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:56.849787 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a246b-5669-40a7-ac44-46191e5bbe8d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:56.850772 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 396/782] 10.222.0.69 () {56 vars in 1087 bytes} [Thu Jul 14 18:49:56 2022] GET /identity/v3/services => generated 772 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:49:56.906319 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3132e43-b159-4be2-8abf-4c9075bbd411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:56.907048 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3132e43-b159-4be2-8abf-4c9075bbd411 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:56.907360 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3132e43-b159-4be2-8abf-4c9075bbd411 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:56.907636 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3132e43-b159-4be2-8abf-4c9075bbd411 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:56.925375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3132e43-b159-4be2-8abf-4c9075bbd411 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:56.942353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b3132e43-b159-4be2-8abf-4c9075bbd411 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:49:56.942353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:56.942353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:56.942353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:56.942353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:56.942353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:56.942353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:56.942353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:56.942353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:56.942353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:56.942353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:56.942353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:56.942353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:56.942353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:56.942353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:56.942353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:56.943588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:56.943588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:56.943588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:56.943588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:56.943588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:56.943588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:56.943588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:56.943588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:56.943588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:56.943588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:56.943588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:56.943588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:56.943588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:56.943588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:56.943588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:56.943588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:56.944658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:56.944658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:56.944658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:56.944658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:56.944658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:56.944658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:56.944658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:56.944658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:56.944658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:56.944658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:56.944658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:49:56.944658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:56.944658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 387/783] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:49:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:56.950055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c10f5d-ed24-48c7-8043-0b0a67371599 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:57.122910 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c10f5d-ed24-48c7-8043-0b0a67371599 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:57.159899 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e68612-96b2-4acf-87dc-a477fbd37645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:57.160507 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e68612-96b2-4acf-87dc-a477fbd37645 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:57.160790 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e68612-96b2-4acf-87dc-a477fbd37645 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:57.161069 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e68612-96b2-4acf-87dc-a477fbd37645 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:57.161811 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e68612-96b2-4acf-87dc-a477fbd37645 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:57.162118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e68612-96b2-4acf-87dc-a477fbd37645 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:57.163512 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e68612-96b2-4acf-87dc-a477fbd37645 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:57.172580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 397/784] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:57.178261 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3132e43-b159-4be2-8abf-4c9075bbd411 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:57.266727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:57.282865 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3132e43-b159-4be2-8abf-4c9075bbd411 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:57.325787 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8656023-c4d5-4ebb-9664-0a7fc1e40cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:57.326591 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8656023-c4d5-4ebb-9664-0a7fc1e40cef None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:57.326876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8656023-c4d5-4ebb-9664-0a7fc1e40cef None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:57.327148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8656023-c4d5-4ebb-9664-0a7fc1e40cef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:57.337913 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8656023-c4d5-4ebb-9664-0a7fc1e40cef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:57.339399 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8656023-c4d5-4ebb-9664-0a7fc1e40cef None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:57.365643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-f8656023-c4d5-4ebb-9664-0a7fc1e40cef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:57.365643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:57.365643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:57.365643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:57.365643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:57.365643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:57.365643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:57.365643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:57.365643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:57.365643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:57.365643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:57.365643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:57.365643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:57.365643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:57.365643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:57.366588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:57.366588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:57.366588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:57.366588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:57.366588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:57.366588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:57.366588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:57.366588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:57.366588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:57.366588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:57.366588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:57.366588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:57.366588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:57.366588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:57.366588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:57.366588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:57.366588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:57.367534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:57.367534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:57.367534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:57.367534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:57.367534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 388/785] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:49:57.372451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e68612-96b2-4acf-87dc-a477fbd37645 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:57.387532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:57.592442 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b6571-5300-4a2d-95d7-226f69147a38 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:57.592806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b6571-5300-4a2d-95d7-226f69147a38 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:57.593014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b6571-5300-4a2d-95d7-226f69147a38 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:57.597208 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e68612-96b2-4acf-87dc-a477fbd37645 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:57.598386 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 389/786] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:57.602830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274dc58-97ad-4a85-8a31-e67aad3f18d2 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:57.603118 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274dc58-97ad-4a85-8a31-e67aad3f18d2 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:57.603440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274dc58-97ad-4a85-8a31-e67aad3f18d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:57.632645 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b21b2-e12e-45f1-a260-88a02f4e9546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:57.633311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b21b2-e12e-45f1-a260-88a02f4e9546 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:57.633596 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b21b2-e12e-45f1-a260-88a02f4e9546 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:57.633874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b21b2-e12e-45f1-a260-88a02f4e9546 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:57.634581 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b21b2-e12e-45f1-a260-88a02f4e9546 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:57.634886 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b21b2-e12e-45f1-a260-88a02f4e9546 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:57.636176 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b274dc58-97ad-4a85-8a31-e67aad3f18d2 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:57.636292 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b21b2-e12e-45f1-a260-88a02f4e9546 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:57.637385 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b274dc58-97ad-4a85-8a31-e67aad3f18d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:57.649562 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 398/787] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:57.660549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b21b2-e12e-45f1-a260-88a02f4e9546 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:57.738555 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 390/788] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:49:57.773465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274dc58-97ad-4a85-8a31-e67aad3f18d2 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:57.791513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:57.860987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b21b2-e12e-45f1-a260-88a02f4e9546 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:57.899525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:57.918016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7cd51-3c00-4a6e-b33b-53765053a28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:57.919302 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7cd51-3c00-4a6e-b33b-53765053a28d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:57.919828 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7cd51-3c00-4a6e-b33b-53765053a28d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:57.920312 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7cd51-3c00-4a6e-b33b-53765053a28d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:57.930628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7cd51-3c00-4a6e-b33b-53765053a28d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:57.932497 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7cd51-3c00-4a6e-b33b-53765053a28d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:57.948531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-85f7cd51-3c00-4a6e-b33b-53765053a28d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 18:49:57.948531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:57.948531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:57.948531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:57.948531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:57.948531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:57.948531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:57.948531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:57.948531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:57.948531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:57.948531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:57.948531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:57.948531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:57.948531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:57.948531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:57.948531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:57.949526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:57.949526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:57.949526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:57.949526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:57.949526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:57.949526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:57.949526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:57.949526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:57.949526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:57.949526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:57.949526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:57.949526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:57.949526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:57.949526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:57.949526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:57.949526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:57.949526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:57.949526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:57.950467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:57.950467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:57.950467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:57.950467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:57.950467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:57.950467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:57.950467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:57.950467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:57.950467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 18:49:57.950467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:57.953423 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 399/789] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:49:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:57.959197 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7cd51-3c00-4a6e-b33b-53765053a28d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:57.969907 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274dc58-97ad-4a85-8a31-e67aad3f18d2 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:58.051494 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f208212-58f2-44da-aa53-08bd601d5014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:58.052339 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f208212-58f2-44da-aa53-08bd601d5014 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:58.052627 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f208212-58f2-44da-aa53-08bd601d5014 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:58.052904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f208212-58f2-44da-aa53-08bd601d5014 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:58.053565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f208212-58f2-44da-aa53-08bd601d5014 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:58.054937 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f208212-58f2-44da-aa53-08bd601d5014 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:58.070449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-8f208212-58f2-44da-aa53-08bd601d5014 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 18:49:58.070449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:58.070449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:49:58.070449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:49:58.070449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:49:58.070449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:49:58.070449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:49:58.070449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:49:58.070449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 18:49:58.070449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:58.070449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:49:58.070449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:58.070449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:58.070449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:58.070449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:58.070449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:58.071558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:58.071558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:58.071558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:58.071558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:58.071558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:58.071558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:58.071558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:58.071558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:49:58.071558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:49:58.071558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:49:58.071558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:49:58.071558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:58.071558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:58.071558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:58.071558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:58.071558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:58.072378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:58.072378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:58.072378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:58.072378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:58.072378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:58.072378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:58.072378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:58.072378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:58.072378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:58.072378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:58.072378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:58.072378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:49:58.072378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:49:58.072378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 18:49:58.072378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:58.073168 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 391/790] 10.222.0.69 () {56 vars in 1129 bytes} [Thu Jul 14 18:49:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:58.074577 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096c68f-be51-45c8-9e2c-4998545220b3 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:58.074577 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096c68f-be51-45c8-9e2c-4998545220b3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:58.074774 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096c68f-be51-45c8-9e2c-4998545220b3 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:58.079259 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 392/791] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:58.080112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096c68f-be51-45c8-9e2c-4998545220b3 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:58.108672 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7cd51-3c00-4a6e-b33b-53765053a28d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:58.135583 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:58.145825 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921861a-3c0b-4fb1-bd65-06d3515d73be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:58.150058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921861a-3c0b-4fb1-bd65-06d3515d73be None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:58.150058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921861a-3c0b-4fb1-bd65-06d3515d73be None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:58.150058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921861a-3c0b-4fb1-bd65-06d3515d73be None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:58.150058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921861a-3c0b-4fb1-bd65-06d3515d73be None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:58.150058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921861a-3c0b-4fb1-bd65-06d3515d73be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:58.150058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921861a-3c0b-4fb1-bd65-06d3515d73be None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:58.173759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 400/792] 10.222.0.69 () {56 vars in 1135 bytes} [Thu Jul 14 18:49:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:58.175419 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29134c-ce53-4f0b-af4f-73c4fc8b224b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:58.175872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29134c-ce53-4f0b-af4f-73c4fc8b224b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:58.176156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29134c-ce53-4f0b-af4f-73c4fc8b224b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:58.192791 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9b29134c-ce53-4f0b-af4f-73c4fc8b224b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:58.193968 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9b29134c-ce53-4f0b-af4f-73c4fc8b224b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:58.208042 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096c68f-be51-45c8-9e2c-4998545220b3 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:58.245870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:58.254067 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf385ca-6cc6-456a-83e5-0fc8172f356f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:58.256150 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf385ca-6cc6-456a-83e5-0fc8172f356f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:58.256150 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf385ca-6cc6-456a-83e5-0fc8172f356f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:58.256829 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf385ca-6cc6-456a-83e5-0fc8172f356f None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:58.257948 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf385ca-6cc6-456a-83e5-0fc8172f356f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:58.258407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf385ca-6cc6-456a-83e5-0fc8172f356f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:58.260689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf385ca-6cc6-456a-83e5-0fc8172f356f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:58.275201 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 401/793] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:49:58.275201 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29134c-ce53-4f0b-af4f-73c4fc8b224b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:58.275421 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 393/794] 10.222.0.69 () {56 vars in 1124 bytes} [Thu Jul 14 18:49:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:58.292184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf385ca-6cc6-456a-83e5-0fc8172f356f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:58.384962 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29134c-ce53-4f0b-af4f-73c4fc8b224b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:58.404490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:58.426475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57ebeb-16fa-4e82-b703-c6eb761ac2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:58.427636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57ebeb-16fa-4e82-b703-c6eb761ac2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:58.428193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57ebeb-16fa-4e82-b703-c6eb761ac2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:58.428759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57ebeb-16fa-4e82-b703-c6eb761ac2e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:58.432307 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf385ca-6cc6-456a-83e5-0fc8172f356f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:58.441610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57ebeb-16fa-4e82-b703-c6eb761ac2e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:58.443523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57ebeb-16fa-4e82-b703-c6eb761ac2e1 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:58.463827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-2e57ebeb-16fa-4e82-b703-c6eb761ac2e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:58.463827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:58.463827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:58.463827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:58.463827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:58.463827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:58.463827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:58.463827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:58.463827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:58.463827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:58.463827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:58.463827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:58.463827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:58.463827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:58.463827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:58.464749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:58.464749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:58.464749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:58.464749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:58.464749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:58.464749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:58.464749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:58.464749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:58.464749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:58.464749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:58.464749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:58.464749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:58.464749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:58.464749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:58.464749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:58.464749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:58.464749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:58.465619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:58.465619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:58.465619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:58.465619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:58.468074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 402/795] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:58.469306 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57ebeb-16fa-4e82-b703-c6eb761ac2e1 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:58.472510 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc3d85-8edd-4961-a82c-9ae1732cb392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:58.473200 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc3d85-8edd-4961-a82c-9ae1732cb392 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:58.481268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc3d85-8edd-4961-a82c-9ae1732cb392 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:58.481268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc3d85-8edd-4961-a82c-9ae1732cb392 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:58.481268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc3d85-8edd-4961-a82c-9ae1732cb392 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:58.481268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc3d85-8edd-4961-a82c-9ae1732cb392 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:58.481268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc3d85-8edd-4961-a82c-9ae1732cb392 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:58.482519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 394/796] 10.222.0.69 () {56 vars in 1124 bytes} [Thu Jul 14 18:49:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:58.483191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc3d85-8edd-4961-a82c-9ae1732cb392 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:58.587280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:58.615858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57ebeb-16fa-4e82-b703-c6eb761ac2e1 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:58.642845 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc3d85-8edd-4961-a82c-9ae1732cb392 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:58.655307 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff64dd-5a76-4467-967b-8290dfc80777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:58.656048 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff64dd-5a76-4467-967b-8290dfc80777 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:58.656334 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff64dd-5a76-4467-967b-8290dfc80777 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:58.656611 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff64dd-5a76-4467-967b-8290dfc80777 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:58.667746 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff64dd-5a76-4467-967b-8290dfc80777 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:58.676874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-99ff64dd-5a76-4467-967b-8290dfc80777 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:58.676874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:58.676874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:58.676874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:58.676874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:58.676874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:58.676874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:58.676874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:58.676874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:58.676874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:58.676874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:58.676874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:58.676874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:58.676874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:58.676874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:58.676874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:58.678076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:58.678076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:58.678076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:58.678076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:58.678076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:58.678076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:58.678076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:58.678076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:58.678076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:58.678076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:58.678076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:58.678076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:58.678076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:58.678076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:58.678076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:58.678076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:58.678925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:58.678925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:58.678925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:58.678925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:58.678925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:58.678925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:58.678925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:58.678925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:58.678925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:58.678925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:58.678925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:49:58.678925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:58.678925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 403/797] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:49:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:58.679654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff64dd-5a76-4467-967b-8290dfc80777 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:58.688419 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113bdc4-d2c5-4fba-b8a2-dda47d1e67d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:58.689838 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113bdc4-d2c5-4fba-b8a2-dda47d1e67d4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:58.690865 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113bdc4-d2c5-4fba-b8a2-dda47d1e67d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:58.691363 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113bdc4-d2c5-4fba-b8a2-dda47d1e67d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:58.692718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113bdc4-d2c5-4fba-b8a2-dda47d1e67d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:58.693512 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113bdc4-d2c5-4fba-b8a2-dda47d1e67d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:58.695461 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113bdc4-d2c5-4fba-b8a2-dda47d1e67d4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:58.711459 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 395/798] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:58.712355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113bdc4-d2c5-4fba-b8a2-dda47d1e67d4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:58.713214 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:58.802050 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff64dd-5a76-4467-967b-8290dfc80777 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:58.817075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113bdc4-d2c5-4fba-b8a2-dda47d1e67d4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:58.834694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:58.840243 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb61295-7a87-4967-b4a5-dcc99c32f684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:58.841176 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb61295-7a87-4967-b4a5-dcc99c32f684 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:58.841703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb61295-7a87-4967-b4a5-dcc99c32f684 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:58.842156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb61295-7a87-4967-b4a5-dcc99c32f684 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:58.843482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb61295-7a87-4967-b4a5-dcc99c32f684 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:58.844865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb61295-7a87-4967-b4a5-dcc99c32f684 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:58.852608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41267b9c-9520-4df5-bd9c-510fe6590dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:58.853278 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41267b9c-9520-4df5-bd9c-510fe6590dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:58.853634 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41267b9c-9520-4df5-bd9c-510fe6590dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:58.853864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41267b9c-9520-4df5-bd9c-510fe6590dd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:58.854612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41267b9c-9520-4df5-bd9c-510fe6590dd1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:58.854920 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41267b9c-9520-4df5-bd9c-510fe6590dd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:58.856873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41267b9c-9520-4df5-bd9c-510fe6590dd1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:58.866459 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 396/799] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:49:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:58.869695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41267b9c-9520-4df5-bd9c-510fe6590dd1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:58.951599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 404/800] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 14 18:49:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:49:58.953413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76b671-fc03-404d-89d3-3d3c59e33954 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:58.953799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76b671-fc03-404d-89d3-3d3c59e33954 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:58.954174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76b671-fc03-404d-89d3-3d3c59e33954 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:58.955823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 405/801] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:58.956510 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e76b671-fc03-404d-89d3-3d3c59e33954 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:58.982698 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41267b9c-9520-4df5-bd9c-510fe6590dd1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:59.065975 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b295c9-733a-467d-886b-1aab46bbabe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:59.067844 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b295c9-733a-467d-886b-1aab46bbabe9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:59.068795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b295c9-733a-467d-886b-1aab46bbabe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:59.069295 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b295c9-733a-467d-886b-1aab46bbabe9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:59.074159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:59.080872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b295c9-733a-467d-886b-1aab46bbabe9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:59.083635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b295c9-733a-467d-886b-1aab46bbabe9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:59.095875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e76b671-fc03-404d-89d3-3d3c59e33954 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:59.103682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-a6b295c9-733a-467d-886b-1aab46bbabe9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:59.103682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:59.103682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:59.103682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:59.103682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:59.103682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:59.103682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:59.103682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:59.103682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:59.103682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:59.103682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:59.103682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:59.103682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:49:59.103682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:49:59.103682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:49:59.104610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:49:59.104610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:59.104610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:59.104610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:59.104610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:59.104610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:59.104610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:59.104610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:59.104610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:59.104610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:59.104610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:59.104610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:59.104610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:59.104610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:59.104610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:59.104610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:59.104610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:59.105576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:49:59.105576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:49:59.105576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:49:59.105576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:49:59.105576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:49:59.105576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:49:59.105576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:49:59.105576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:49:59.109903 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 397/802] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:59.111200 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c182d-75b9-40ca-a527-5f6d54cf268b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:59.111491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c182d-75b9-40ca-a527-5f6d54cf268b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:59.111770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c182d-75b9-40ca-a527-5f6d54cf268b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:59.133162 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-5b1c182d-75b9-40ca-a527-5f6d54cf268b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:59.134065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5b1c182d-75b9-40ca-a527-5f6d54cf268b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:59.156456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d1389-9bba-4131-ba02-d019c0a617ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:59.157572 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d1389-9bba-4131-ba02-d019c0a617ac None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:59.157861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d1389-9bba-4131-ba02-d019c0a617ac None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:59.158140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d1389-9bba-4131-ba02-d019c0a617ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:59.171547 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:59.174209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d1389-9bba-4131-ba02-d019c0a617ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:59.177600 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d1389-9bba-4131-ba02-d019c0a617ac None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:59.194381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-2f0d1389-9bba-4131-ba02-d019c0a617ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:59.194381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:59.194381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:59.194381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:59.194381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:59.194381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:59.194381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:59.194381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:59.194381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:59.194381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:59.194381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:59.194381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:59.194381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:49:59.194381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:49:59.194381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:49:59.195622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:49:59.195622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:59.195622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:59.195622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:59.195622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:59.195622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:59.195622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:59.195622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:59.195622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:59.195622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:59.195622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:59.195622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:59.195622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:59.195622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:59.195622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:59.195622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:59.195622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:59.196517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:49:59.196517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:49:59.196517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:49:59.196517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:59.196517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 406/803] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:59.197330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d1389-9bba-4131-ba02-d019c0a617ac None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:59.232259 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 398/804] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:49:59.234355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c182d-75b9-40ca-a527-5f6d54cf268b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:59.316347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d1389-9bba-4131-ba02-d019c0a617ac None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:59.353953 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547dd01-46eb-49d9-836a-df794171ed78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:59.354624 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547dd01-46eb-49d9-836a-df794171ed78 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:59.354961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547dd01-46eb-49d9-836a-df794171ed78 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:59.355281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547dd01-46eb-49d9-836a-df794171ed78 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:59.355396 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c182d-75b9-40ca-a527-5f6d54cf268b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:59.356123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547dd01-46eb-49d9-836a-df794171ed78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:59.356438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547dd01-46eb-49d9-836a-df794171ed78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:59.358065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547dd01-46eb-49d9-836a-df794171ed78 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:59.359724 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:59.376165 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 407/805] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:49:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:59.377156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547dd01-46eb-49d9-836a-df794171ed78 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:59.394310 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544a838-0db1-45c8-884a-b793b5672a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:59.395272 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544a838-0db1-45c8-884a-b793b5672a1b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:59.395730 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544a838-0db1-45c8-884a-b793b5672a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:59.396243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544a838-0db1-45c8-884a-b793b5672a1b None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:59.397166 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544a838-0db1-45c8-884a-b793b5672a1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:59.397624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544a838-0db1-45c8-884a-b793b5672a1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:59.399137 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544a838-0db1-45c8-884a-b793b5672a1b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:59.413401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 399/806] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:59.414624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544a838-0db1-45c8-884a-b793b5672a1b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:59.445238 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:59.489402 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547dd01-46eb-49d9-836a-df794171ed78 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:59.522955 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544a838-0db1-45c8-884a-b793b5672a1b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:59.527730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1fdd65-e47c-420a-aa60-90db0b870b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:59.528571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fdd65-e47c-420a-aa60-90db0b870b97 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:59.528853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fdd65-e47c-420a-aa60-90db0b870b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:59.529127 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fdd65-e47c-420a-aa60-90db0b870b97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:59.540423 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fdd65-e47c-420a-aa60-90db0b870b97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:49:59.548363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-bf1fdd65-e47c-420a-aa60-90db0b870b97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:49:59.548363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:59.548363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:59.548363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:59.548363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:59.548363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:59.548363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:59.548363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:59.548363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:59.548363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:59.548363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:59.548363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:59.548363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:49:59.548363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:49:59.548363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:49:59.548363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:49:59.549519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:49:59.549519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:49:59.549519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:49:59.549519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:49:59.549519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:59.549519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:59.549519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:59.549519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:59.549519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:59.549519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:59.549519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:59.549519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:59.549519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:59.549519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:59.549519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:59.549519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:59.550556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:59.550556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:59.550556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:59.550556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:59.550556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:49:59.550556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:49:59.550556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:49:59.550556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:49:59.550556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:49:59.550556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:49:59.550556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:49:59.550556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:59.554612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 408/807] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:49:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:59.555484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1fdd65-e47c-420a-aa60-90db0b870b97 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:59.563990 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c3468-dfdd-4f4f-967e-bd95ec1e3a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:59.564885 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c3468-dfdd-4f4f-967e-bd95ec1e3a39 None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:59.565762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c3468-dfdd-4f4f-967e-bd95ec1e3a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:59.566038 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c3468-dfdd-4f4f-967e-bd95ec1e3a39 None admin] PATH_INFO: `/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/4368f68724f946eca53084a9ec7ff590/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:59.568054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c3468-dfdd-4f4f-967e-bd95ec1e3a39 None admin] RBAC: Authorizing `identity:create_grant(project_id=6003dba8ddd247e7842d28ea7ca5238c, user_id=4368f68724f946eca53084a9ec7ff590, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:59.569540 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c3468-dfdd-4f4f-967e-bd95ec1e3a39 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:59.610221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:59.653816 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1fdd65-e47c-420a-aa60-90db0b870b97 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:59.686732 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32df86d-228a-4472-be7c-88f952f6b98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:59.687836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32df86d-228a-4472-be7c-88f952f6b98e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:59.688123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32df86d-228a-4472-be7c-88f952f6b98e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:59.688403 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32df86d-228a-4472-be7c-88f952f6b98e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:59.694672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 400/808] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:49:59 2022] PUT /identity/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/4368f68724f946eca53084a9ec7ff590/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:49:59.694672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e62db-674d-44e9-a0bf-de30fc1da396 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:59.694672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e62db-674d-44e9-a0bf-de30fc1da396 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:59.694672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e62db-674d-44e9-a0bf-de30fc1da396 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:59.694672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 401/809] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:49:59.694672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e62db-674d-44e9-a0bf-de30fc1da396 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:59.706807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32df86d-228a-4472-be7c-88f952f6b98e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:59.708617 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32df86d-228a-4472-be7c-88f952f6b98e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:59.725346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c32df86d-228a-4472-be7c-88f952f6b98e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 18:49:59.725346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:49:59.725346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:49:59.725346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:49:59.725346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:49:59.725346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:49:59.725346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:49:59.725346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:49:59.725346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:49:59.725346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:49:59.725346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:49:59.725346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:49:59.725346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:49:59.725346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:49:59.725346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:49:59.725346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:49:59.726523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:49:59.726523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:49:59.726523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:49:59.726523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:59.726523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:49:59.726523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:49:59.726523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:49:59.726523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:49:59.726523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:49:59.726523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:49:59.726523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:49:59.726523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:49:59.726523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:49:59.726523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:49:59.726523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:49:59.726523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:49:59.726523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:49:59.726523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:49:59.727730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:49:59.727730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:49:59.727730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:49:59.727730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:49:59.727730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:49:59.727730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:49:59.727730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:49:59.727730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:49:59.727730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 18:49:59.727730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:49:59.729812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 409/810] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 14 18:49:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:49:59.731321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c0150-bb38-431f-ab69-51958dd451ba None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:59.731766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c0150-bb38-431f-ab69-51958dd451ba None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:59.732224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c0150-bb38-431f-ab69-51958dd451ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:59.751039 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-a82c0150-bb38-431f-ab69-51958dd451ba None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:49:59.753299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a82c0150-bb38-431f-ab69-51958dd451ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:49:59.825401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:59.843563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e62db-674d-44e9-a0bf-de30fc1da396 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:49:59.859240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 410/811] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:49:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:49:59.859240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c0150-bb38-431f-ab69-51958dd451ba None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:59.895236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a622262-d401-4872-872d-a10c2b43c7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:49:59.896198 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a622262-d401-4872-872d-a10c2b43c7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:49:59.896710 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a622262-d401-4872-872d-a10c2b43c7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:49:59.897353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a622262-d401-4872-872d-a10c2b43c7b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:49:59.898107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a622262-d401-4872-872d-a10c2b43c7b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:49:59.898416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a622262-d401-4872-872d-a10c2b43c7b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:49:59.900784 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a622262-d401-4872-872d-a10c2b43c7b2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:49:59.912961 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 402/812] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:49:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:49:59.913130 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:49:59.913229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a622262-d401-4872-872d-a10c2b43c7b2 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:49:59.975425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c0150-bb38-431f-ab69-51958dd451ba None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:00.029556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a437246-9482-422f-89da-846e917c0ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:00.030548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a437246-9482-422f-89da-846e917c0ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:00.030961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a437246-9482-422f-89da-846e917c0ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:00.031403 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a437246-9482-422f-89da-846e917c0ff6 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:00.032286 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a437246-9482-422f-89da-846e917c0ff6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:00.032708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a437246-9482-422f-89da-846e917c0ff6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:00.034887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a437246-9482-422f-89da-846e917c0ff6 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:00.039405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a622262-d401-4872-872d-a10c2b43c7b2 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:00.062493 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 411/813] 10.222.0.69 () {56 vars in 1149 bytes} [Thu Jul 14 18:49:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:00.063433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a437246-9482-422f-89da-846e917c0ff6 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:00.077353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d45b02-dee1-412f-8326-d571a79848b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:00.078434 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d45b02-dee1-412f-8326-d571a79848b9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:00.078818 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d45b02-dee1-412f-8326-d571a79848b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:00.079295 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d45b02-dee1-412f-8326-d571a79848b9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:00.080056 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d45b02-dee1-412f-8326-d571a79848b9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:00.081891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d45b02-dee1-412f-8326-d571a79848b9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:00.095106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-16d45b02-dee1-412f-8326-d571a79848b9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 18:50:00.095106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:00.095106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:50:00.095106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:50:00.095106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:50:00.095106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:50:00.095106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:50:00.095106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:00.095106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 18:50:00.095106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:00.095106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:00.095106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:00.095106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:00.095106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:00.095106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:00.095106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:00.095106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:00.096184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:00.096184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:00.096184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:00.096184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:00.096184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:00.096184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:00.096184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:50:00.096184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:50:00.096184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:50:00.096184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:50:00.096184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:00.096184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:00.096184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:00.096184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:00.096184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:00.096184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:00.097475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:00.097475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:00.097475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:00.097475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:00.097475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:00.097475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:00.097475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:00.097475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:00.097475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:00.097475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:00.097475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:50:00.097475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:00.097475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 18:50:00.097475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:00.101774 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 403/814] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:49:59 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:00.102724 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d45b02-dee1-412f-8326-d571a79848b9 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:00.121505 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:00.187062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a437246-9482-422f-89da-846e917c0ff6 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:00.188714 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:00.217637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d45b02-dee1-412f-8326-d571a79848b9 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:00.230897 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b998d8a-cdd3-4d93-9b91-c4f1d3e584a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:00.231731 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b998d8a-cdd3-4d93-9b91-c4f1d3e584a9 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:00.232174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b998d8a-cdd3-4d93-9b91-c4f1d3e584a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:00.232458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b998d8a-cdd3-4d93-9b91-c4f1d3e584a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:00.245645 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b998d8a-cdd3-4d93-9b91-c4f1d3e584a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:00.247763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b998d8a-cdd3-4d93-9b91-c4f1d3e584a9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:00.255800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48162a93-bf97-4f60-94ab-e8eef03be8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:00.257090 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48162a93-bf97-4f60-94ab-e8eef03be8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:00.257635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48162a93-bf97-4f60-94ab-e8eef03be8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:00.258105 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48162a93-bf97-4f60-94ab-e8eef03be8c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:00.268188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-5b998d8a-cdd3-4d93-9b91-c4f1d3e584a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:00.268188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:00.268188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:00.268188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:00.268188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:00.268188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:00.268188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:00.268188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:00.268188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:00.268188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:00.268188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:00.268188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:00.268188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:00.268188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:00.268188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:00.269291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:00.269291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:00.269291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:00.269291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:00.269291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:00.269291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:00.269291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:00.269291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:00.269291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:00.269291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:00.269291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:00.269291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:00.269291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:00.269291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:00.269291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:00.269291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:00.269291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:00.270255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:00.270255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:00.270255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:00.270255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:00.270255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 412/815] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:00.270541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b998d8a-cdd3-4d93-9b91-c4f1d3e584a9 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:00.272405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48162a93-bf97-4f60-94ab-e8eef03be8c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:00.274385 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48162a93-bf97-4f60-94ab-e8eef03be8c8 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:00.294607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-48162a93-bf97-4f60-94ab-e8eef03be8c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:00.294607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:00.294607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:00.294607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:00.294607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:00.294607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:00.294607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:00.294607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:00.294607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:00.294607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:00.294607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:00.294607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:00.294607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:00.294607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:00.294607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:00.295728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:00.295728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:00.295728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:00.295728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:00.295728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:00.295728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:00.295728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:00.295728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:00.295728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:00.295728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:00.295728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:00.295728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:00.295728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:00.295728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:00.295728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:00.295728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:00.295728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:00.296692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:00.296692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:00.296692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:00.296692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:00.298831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 404/816] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:00.300421 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48162a93-bf97-4f60-94ab-e8eef03be8c8 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:00.368385 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:00.397255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b998d8a-cdd3-4d93-9b91-c4f1d3e584a9 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:00.410916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48162a93-bf97-4f60-94ab-e8eef03be8c8 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:00.439835 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098703e0-69a5-4270-a8b4-38bd177c2f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:00.440954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098703e0-69a5-4270-a8b4-38bd177c2f96 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:00.441248 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098703e0-69a5-4270-a8b4-38bd177c2f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:00.441527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098703e0-69a5-4270-a8b4-38bd177c2f96 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:00.442514 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098703e0-69a5-4270-a8b4-38bd177c2f96 None admin] RBAC: Adding query filter params (name=network) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:00.442829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098703e0-69a5-4270-a8b4-38bd177c2f96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:00.444425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098703e0-69a5-4270-a8b4-38bd177c2f96 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:00.454384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097eba7-1c82-482c-99c3-ba27232f4121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:00.454601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 413/817] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 14 18:50:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:00.454601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098703e0-69a5-4270-a8b4-38bd177c2f96 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:00.455942 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097eba7-1c82-482c-99c3-ba27232f4121 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:00.456379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097eba7-1c82-482c-99c3-ba27232f4121 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:00.456805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097eba7-1c82-482c-99c3-ba27232f4121 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:00.457685 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097eba7-1c82-482c-99c3-ba27232f4121 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:00.458207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097eba7-1c82-482c-99c3-ba27232f4121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:00.459867 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097eba7-1c82-482c-99c3-ba27232f4121 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:00.469915 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:00.476765 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 405/818] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:00.477952 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097eba7-1c82-482c-99c3-ba27232f4121 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:00.558585 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098703e0-69a5-4270-a8b4-38bd177c2f96 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:00.591831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097eba7-1c82-482c-99c3-ba27232f4121 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:00.593233 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5bb6e-3c79-4ee8-913d-e7b7acfa048c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:00.593895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5bb6e-3c79-4ee8-913d-e7b7acfa048c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:00.594180 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5bb6e-3c79-4ee8-913d-e7b7acfa048c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:00.594458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5bb6e-3c79-4ee8-913d-e7b7acfa048c None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:00.595843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5bb6e-3c79-4ee8-913d-e7b7acfa048c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:00.595843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5bb6e-3c79-4ee8-913d-e7b7acfa048c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:00.597390 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5bb6e-3c79-4ee8-913d-e7b7acfa048c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:00.613985 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 414/819] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:00.614886 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5bb6e-3c79-4ee8-913d-e7b7acfa048c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:00.620861 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:00.633188 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a0a5f-5950-41ad-9931-6d5ed7839b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:00.634641 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a0a5f-5950-41ad-9931-6d5ed7839b48 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:00.635070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a0a5f-5950-41ad-9931-6d5ed7839b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:00.635533 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a0a5f-5950-41ad-9931-6d5ed7839b48 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:00.636614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a0a5f-5950-41ad-9931-6d5ed7839b48 None admin] RBAC: Adding query filter params (type=network) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:00.637076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a0a5f-5950-41ad-9931-6d5ed7839b48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:00.638631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a0a5f-5950-41ad-9931-6d5ed7839b48 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:00.649313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 406/820] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 14 18:50:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:00.650357 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a0a5f-5950-41ad-9931-6d5ed7839b48 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:00.693295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:00.714754 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5bb6e-3c79-4ee8-913d-e7b7acfa048c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:00.749309 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde668d0-45e4-49be-8adf-ac3c3fcf21a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:00.750110 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde668d0-45e4-49be-8adf-ac3c3fcf21a4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:00.750396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde668d0-45e4-49be-8adf-ac3c3fcf21a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:00.750668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde668d0-45e4-49be-8adf-ac3c3fcf21a4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:00.763358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde668d0-45e4-49be-8adf-ac3c3fcf21a4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:00.764818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde668d0-45e4-49be-8adf-ac3c3fcf21a4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:00.767077 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a0a5f-5950-41ad-9931-6d5ed7839b48 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:00.786074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-bde668d0-45e4-49be-8adf-ac3c3fcf21a4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 18:50:00.786074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:00.786074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:00.786074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:00.786074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:00.786074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:00.786074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:00.786074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:00.786074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:00.786074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:00.786074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:00.786074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:00.786074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:50:00.786074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:50:00.786074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:50:00.786074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:50:00.787238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:00.787238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:00.787238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:50:00.787238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:00.787238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:50:00.787238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:00.787238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:00.787238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:00.787238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:00.787238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:00.787238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:00.787238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:00.787238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:00.787238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:00.787238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:00.787238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:00.787238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:00.787238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:00.788294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:00.788294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:00.788294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:50:00.788294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:50:00.788294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:50:00.788294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:50:00.788294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:50:00.788294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:50:00.788294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 18:50:00.788294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:00.788294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 415/821] 10.222.0.69 () {56 vars in 1093 bytes} [Thu Jul 14 18:50:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:00.788875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde668d0-45e4-49be-8adf-ac3c3fcf21a4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:00.809771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e542e80-6593-46cc-b760-7e8d3a7cfade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:00.810919 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e542e80-6593-46cc-b760-7e8d3a7cfade None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:00.811405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e542e80-6593-46cc-b760-7e8d3a7cfade None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:00.811818 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e542e80-6593-46cc-b760-7e8d3a7cfade None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:00.820041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e542e80-6593-46cc-b760-7e8d3a7cfade None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:00.822060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e542e80-6593-46cc-b760-7e8d3a7cfade None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:00.838593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-2e542e80-6593-46cc-b760-7e8d3a7cfade None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:00.838593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:00.838593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:00.838593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:00.838593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:00.838593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:00.838593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:00.838593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:00.838593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:00.838593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:00.838593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:00.838593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:00.838593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:50:00.838593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:50:00.838593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:50:00.839692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:50:00.839692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:00.839692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:00.839692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:00.839692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:00.839692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:00.839692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:00.839692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:00.839692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:00.839692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:00.839692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:00.839692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:00.839692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:00.839692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:00.839692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:00.839692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:00.839692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:00.840597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:50:00.840597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:50:00.840597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:50:00.840597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:50:00.840597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:50:00.840597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:50:00.840597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:00.840597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:00.842729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 407/822] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:50:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:00.843689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e542e80-6593-46cc-b760-7e8d3a7cfade None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:00.895973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde668d0-45e4-49be-8adf-ac3c3fcf21a4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:00.898868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:00.933380 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8939991-f4e4-4669-8d40-5ab36baafe29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:00.934093 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8939991-f4e4-4669-8d40-5ab36baafe29 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:00.934384 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8939991-f4e4-4669-8d40-5ab36baafe29 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:00.934666 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8939991-f4e4-4669-8d40-5ab36baafe29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:00.935467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8939991-f4e4-4669-8d40-5ab36baafe29 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:00.936852 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8939991-f4e4-4669-8d40-5ab36baafe29 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:00.944343 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:00.961615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e542e80-6593-46cc-b760-7e8d3a7cfade None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:01.003195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47c7f9-e553-416f-878d-c1052be774ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:01.003890 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47c7f9-e553-416f-878d-c1052be774ef None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:01.004182 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47c7f9-e553-416f-878d-c1052be774ef None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:01.004460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47c7f9-e553-416f-878d-c1052be774ef None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:01.005867 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47c7f9-e553-416f-878d-c1052be774ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:01.005867 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47c7f9-e553-416f-878d-c1052be774ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:01.007092 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47c7f9-e553-416f-878d-c1052be774ef None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:01.030490 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 408/823] 10.222.0.69 () {56 vars in 1133 bytes} [Thu Jul 14 18:50:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:01.030490 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47c7f9-e553-416f-878d-c1052be774ef None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:01.039554 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 416/824] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 14 18:50:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:50:01.041202 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8939991-f4e4-4669-8d40-5ab36baafe29 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:01.177053 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47c7f9-e553-416f-878d-c1052be774ef None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:01.180939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8939991-f4e4-4669-8d40-5ab36baafe29 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:01.188273 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:01.223502 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874c604-a2d4-4804-8c42-2df25e344361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:01.224535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874c604-a2d4-4804-8c42-2df25e344361 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:01.224885 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874c604-a2d4-4804-8c42-2df25e344361 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:01.225232 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874c604-a2d4-4804-8c42-2df25e344361 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:01.226128 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874c604-a2d4-4804-8c42-2df25e344361 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:01.226509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874c604-a2d4-4804-8c42-2df25e344361 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:01.227614 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08432f-c1e1-4d24-a02c-33ae7525afa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:01.228134 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874c604-a2d4-4804-8c42-2df25e344361 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:01.228406 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08432f-c1e1-4d24-a02c-33ae7525afa4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:01.228757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08432f-c1e1-4d24-a02c-33ae7525afa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:01.229105 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08432f-c1e1-4d24-a02c-33ae7525afa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:01.244365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 409/825] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:50:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:01.245919 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b86aa-f278-4a7a-b037-8fc4ce4a9958 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:01.246314 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b86aa-f278-4a7a-b037-8fc4ce4a9958 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:01.246833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b86aa-f278-4a7a-b037-8fc4ce4a9958 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:01.248538 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 410/826] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:01.252074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b86aa-f278-4a7a-b037-8fc4ce4a9958 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:01.311150 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:01.327122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08432f-c1e1-4d24-a02c-33ae7525afa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:01.329329 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08432f-c1e1-4d24-a02c-33ae7525afa4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:01.373955 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b86aa-f278-4a7a-b037-8fc4ce4a9958 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:01.410597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324aeb0-64c4-414b-9b77-52eef542331e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:01.411517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324aeb0-64c4-414b-9b77-52eef542331e None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:01.411800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324aeb0-64c4-414b-9b77-52eef542331e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:01.412074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324aeb0-64c4-414b-9b77-52eef542331e None admin] PATH_INFO: `/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/4ef0fad9a71d43bfbaba9c87dfed41c4/roles/24846e82841b4b05b4d9e9d193a98868` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:01.427889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324aeb0-64c4-414b-9b77-52eef542331e None admin] RBAC: Authorizing `identity:create_grant(project_id=6003dba8ddd247e7842d28ea7ca5238c, user_id=4ef0fad9a71d43bfbaba9c87dfed41c4, role_id=24846e82841b4b05b4d9e9d193a98868)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:01.429262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324aeb0-64c4-414b-9b77-52eef542331e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:01.441461 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 417/827] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:50:01 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:50:01.442865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9cb5b-dffa-4807-a61f-b450ceeab98d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:01.445347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9cb5b-dffa-4807-a61f-b450ceeab98d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:01.445347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9cb5b-dffa-4807-a61f-b450ceeab98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:01.460059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-70a9cb5b-dffa-4807-a61f-b450ceeab98d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:01.461239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-70a9cb5b-dffa-4807-a61f-b450ceeab98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:01.496917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 411/828] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:50:01 2022] PUT /identity/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/4ef0fad9a71d43bfbaba9c87dfed41c4/roles/24846e82841b4b05b4d9e9d193a98868 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:50:01.498023 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324aeb0-64c4-414b-9b77-52eef542331e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:01.532653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:01.546020 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 418/829] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:01.574423 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9cb5b-dffa-4807-a61f-b450ceeab98d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:01.599131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:01.633135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324aeb0-64c4-414b-9b77-52eef542331e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:01.671437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08172ead-1b36-4c6a-b7ab-633563bc2610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:01.672252 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08172ead-1b36-4c6a-b7ab-633563bc2610 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:01.672739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08172ead-1b36-4c6a-b7ab-633563bc2610 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:01.672858 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08172ead-1b36-4c6a-b7ab-633563bc2610 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:01.685024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08172ead-1b36-4c6a-b7ab-633563bc2610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:01.686442 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08172ead-1b36-4c6a-b7ab-633563bc2610 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:01.691723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9cb5b-dffa-4807-a61f-b450ceeab98d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:01.709543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-08172ead-1b36-4c6a-b7ab-633563bc2610 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:01.709543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:01.709543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:01.709543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:01.709543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:01.709543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:01.709543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:01.709543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:01.709543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:01.709543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:01.709543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:01.709543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:01.709543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:01.709543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:01.709543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:01.710653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:01.710653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:01.710653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:01.710653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:01.710653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:01.710653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:01.710653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:01.710653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:01.710653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:01.710653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:01.710653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:01.710653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:01.710653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:01.710653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:01.710653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:01.710653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:01.710653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:01.711769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:01.711769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:01.711769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:01.711769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:01.711769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 412/830] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:01.718240 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08172ead-1b36-4c6a-b7ab-633563bc2610 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:01.734623 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091453e-cc95-40f1-bbfb-d3a4c68d0186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:01.735487 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091453e-cc95-40f1-bbfb-d3a4c68d0186 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:01.735778 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091453e-cc95-40f1-bbfb-d3a4c68d0186 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:01.736051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091453e-cc95-40f1-bbfb-d3a4c68d0186 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:01.736712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091453e-cc95-40f1-bbfb-d3a4c68d0186 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:01.738645 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091453e-cc95-40f1-bbfb-d3a4c68d0186 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:01.754771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-d091453e-cc95-40f1-bbfb-d3a4c68d0186 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 18:50:01.754771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:01.754771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:50:01.754771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:50:01.754771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:50:01.754771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:50:01.754771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:50:01.754771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:01.754771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 18:50:01.754771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:01.754771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:01.754771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:01.754771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:01.754771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:01.754771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:01.754771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:01.754771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:01.756118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:01.756118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:01.756118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:01.756118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:01.756118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:01.756118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:01.756118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:50:01.756118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:50:01.756118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:50:01.756118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:50:01.756118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:01.756118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:01.756118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:01.756118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:01.756118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:01.756118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:01.757114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:01.757114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:01.757114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:01.757114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:01.757114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:01.757114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:01.757114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:01.757114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:01.757114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:01.757114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:01.757114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:50:01.757114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:01.757114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 18:50:01.757114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:01.757114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 419/831] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:50:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:01.763448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091453e-cc95-40f1-bbfb-d3a4c68d0186 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:01.794518 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:01.844934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08172ead-1b36-4c6a-b7ab-633563bc2610 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:01.860638 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:01.877480 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091453e-cc95-40f1-bbfb-d3a4c68d0186 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:01.885329 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262af89-d347-40cb-9fc2-33a5d4d3089d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:01.886016 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262af89-d347-40cb-9fc2-33a5d4d3089d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:01.886306 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262af89-d347-40cb-9fc2-33a5d4d3089d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:01.886588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262af89-d347-40cb-9fc2-33a5d4d3089d None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:01.887372 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262af89-d347-40cb-9fc2-33a5d4d3089d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:01.887709 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262af89-d347-40cb-9fc2-33a5d4d3089d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:01.889103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262af89-d347-40cb-9fc2-33a5d4d3089d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:01.901582 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 413/832] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:01.903325 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bba31-1037-4234-8f67-e048c13d2f7b None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:01.903627 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bba31-1037-4234-8f67-e048c13d2f7b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:01.903908 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bba31-1037-4234-8f67-e048c13d2f7b None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:01.907429 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 414/833] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:01.909309 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2bba31-1037-4234-8f67-e048c13d2f7b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:01.919551 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5c684-ec84-4eb4-848f-996583ad2abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:01.920303 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5c684-ec84-4eb4-848f-996583ad2abb None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:01.920661 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5c684-ec84-4eb4-848f-996583ad2abb None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:01.921009 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5c684-ec84-4eb4-848f-996583ad2abb None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:01.921812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e5c684-ec84-4eb4-848f-996583ad2abb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:01.922189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e5c684-ec84-4eb4-848f-996583ad2abb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:01.923677 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e5c684-ec84-4eb4-848f-996583ad2abb None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:01.933741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 420/834] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:50:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:01.933741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68592349-64ce-47a7-abfe-333062413539 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:01.933741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68592349-64ce-47a7-abfe-333062413539 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:01.933741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68592349-64ce-47a7-abfe-333062413539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:01.947857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-68592349-64ce-47a7-abfe-333062413539 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:01.949061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-68592349-64ce-47a7-abfe-333062413539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:02.025550 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 421/835] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:02.026399 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68592349-64ce-47a7-abfe-333062413539 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:02.047219 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2bba31-1037-4234-8f67-e048c13d2f7b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:02.077441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:02.086005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a3452-8441-4ecf-a9d0-61df3a0a8d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:02.086811 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a3452-8441-4ecf-a9d0-61df3a0a8d1a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:02.087099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a3452-8441-4ecf-a9d0-61df3a0a8d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:02.087443 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a3452-8441-4ecf-a9d0-61df3a0a8d1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:02.094346 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a3452-8441-4ecf-a9d0-61df3a0a8d1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:02.095880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a3452-8441-4ecf-a9d0-61df3a0a8d1a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:02.112740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-bb2a3452-8441-4ecf-a9d0-61df3a0a8d1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:02.112740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:02.112740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:02.112740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:02.112740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:02.112740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:02.112740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:02.112740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:02.112740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:02.112740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:02.112740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:02.112740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:02.112740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:50:02.112740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:50:02.112740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:50:02.113915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:50:02.113915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:02.113915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:02.113915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:02.113915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:02.113915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:02.113915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:02.113915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:02.113915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:02.113915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:02.113915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:02.113915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:02.113915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:02.113915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:02.113915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:02.113915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:02.113915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:02.115127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:50:02.115127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:50:02.115127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:50:02.115127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:50:02.115127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:50:02.115127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:50:02.115127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:02.115127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:02.117688 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 415/836] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:50:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:02.118515 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a3452-8441-4ecf-a9d0-61df3a0a8d1a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:02.122576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:02.152204 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68592349-64ce-47a7-abfe-333062413539 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:02.187597 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f3f65-c89b-4080-ac9e-0b7dba874bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:02.188268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f3f65-c89b-4080-ac9e-0b7dba874bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:02.188557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f3f65-c89b-4080-ac9e-0b7dba874bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:02.188836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f3f65-c89b-4080-ac9e-0b7dba874bb1 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:02.189546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f3f65-c89b-4080-ac9e-0b7dba874bb1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:02.189858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f3f65-c89b-4080-ac9e-0b7dba874bb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:02.191271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f3f65-c89b-4080-ac9e-0b7dba874bb1 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:02.203337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 422/837] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:50:02 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:02.203337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f3f65-c89b-4080-ac9e-0b7dba874bb1 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:02.235339 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a3452-8441-4ecf-a9d0-61df3a0a8d1a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:02.282296 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78132e1-3cbf-471a-b379-446fba29d551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:02.283369 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78132e1-3cbf-471a-b379-446fba29d551 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:02.284611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78132e1-3cbf-471a-b379-446fba29d551 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:02.284916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78132e1-3cbf-471a-b379-446fba29d551 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:02.300078 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78132e1-3cbf-471a-b379-446fba29d551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:50:02.309891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-d78132e1-3cbf-471a-b379-446fba29d551 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:50:02.309891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:02.309891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:02.309891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:02.309891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:02.309891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:02.309891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:02.309891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:02.309891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:02.309891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:02.309891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:02.309891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:02.309891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:50:02.309891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:50:02.309891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:50:02.309891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:50:02.310891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:50:02.310891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:50:02.310891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:50:02.310891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:50:02.310891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:02.310891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:02.310891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:02.310891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:02.310891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:02.310891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:02.310891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:02.310891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:02.310891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:02.310891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:02.310891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:02.310891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:02.311798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:02.311798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:02.311798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:02.311798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:02.311798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:50:02.311798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:50:02.311798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:50:02.311798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:50:02.311798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:50:02.311798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:50:02.311798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:50:02.311798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:02.312431 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 416/838] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:50:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:02.318383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78132e1-3cbf-471a-b379-446fba29d551 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:02.334425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f3f65-c89b-4080-ac9e-0b7dba874bb1 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:02.380882 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053926f-e9b7-4270-870b-c3125f91986b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:02.381718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053926f-e9b7-4270-870b-c3125f91986b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:02.382387 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053926f-e9b7-4270-870b-c3125f91986b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:02.382671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053926f-e9b7-4270-870b-c3125f91986b None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:02.383826 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053926f-e9b7-4270-870b-c3125f91986b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:02.384148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053926f-e9b7-4270-870b-c3125f91986b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:02.384609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:02.386065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053926f-e9b7-4270-870b-c3125f91986b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:02.402444 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 423/839] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:50:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:02.409313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053926f-e9b7-4270-870b-c3125f91986b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:02.412267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:02.453876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78132e1-3cbf-471a-b379-446fba29d551 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:02.503855 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d7934-f06d-4d29-8c92-0e8e75da2cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:02.505002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d7934-f06d-4d29-8c92-0e8e75da2cbf None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:02.505304 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d7934-f06d-4d29-8c92-0e8e75da2cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:02.505588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d7934-f06d-4d29-8c92-0e8e75da2cbf None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:02.506616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d7934-f06d-4d29-8c92-0e8e75da2cbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:02.506933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d7934-f06d-4d29-8c92-0e8e75da2cbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:02.508762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d7934-f06d-4d29-8c92-0e8e75da2cbf None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:02.525877 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 417/840] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:50:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:02.526056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053926f-e9b7-4270-870b-c3125f91986b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:02.528218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d7934-f06d-4d29-8c92-0e8e75da2cbf None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:02.572507 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcac48-f179-4e71-8b3d-8d28fc941eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:02.573187 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcac48-f179-4e71-8b3d-8d28fc941eee None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:02.573480 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcac48-f179-4e71-8b3d-8d28fc941eee None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:02.573760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcac48-f179-4e71-8b3d-8d28fc941eee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:02.574596 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcac48-f179-4e71-8b3d-8d28fc941eee None admin] RBAC: Adding query filter params (role.id=27fc14631c494ca0b2dc7f708be016c2, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=8d27d6df5cb64697adebd961dfbec324) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:02.574910 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcac48-f179-4e71-8b3d-8d28fc941eee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:02.576957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcac48-f179-4e71-8b3d-8d28fc941eee None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:02.591203 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 424/841] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:50:02 2022] GET /identity/v3/role_assignments?user.id=8d27d6df5cb64697adebd961dfbec324&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:02.661313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d7934-f06d-4d29-8c92-0e8e75da2cbf None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:02.673701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:02.705215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8beea0-a6e4-4346-aeb4-33ea5422bf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:02.706535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8beea0-a6e4-4346-aeb4-33ea5422bf27 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:02.706821 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8beea0-a6e4-4346-aeb4-33ea5422bf27 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:02.707096 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8beea0-a6e4-4346-aeb4-33ea5422bf27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:02.720263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8beea0-a6e4-4346-aeb4-33ea5422bf27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:02.721871 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8beea0-a6e4-4346-aeb4-33ea5422bf27 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:02.741496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-5a8beea0-a6e4-4346-aeb4-33ea5422bf27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:02.741496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:02.741496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:02.741496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:02.741496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:02.741496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:02.741496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:02.741496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:02.741496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:02.741496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:02.741496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:02.741496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:02.741496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:02.741496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:02.741496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:02.742556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:02.742556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:02.742556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:02.742556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:02.742556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:02.742556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:02.742556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:02.742556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:02.742556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:02.742556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:02.742556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:02.742556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:02.742556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:02.742556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:02.742556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:02.742556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:02.742556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:02.743469 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:02.743469 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:02.743469 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:02.743469 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:02.743469 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 418/842] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:02.748792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcac48-f179-4e71-8b3d-8d28fc941eee None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:02.830053 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:02.866541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcac48-f179-4e71-8b3d-8d28fc941eee None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:02.902273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bca971-9ad5-4c94-ba88-f134a4023b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:02.902989 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bca971-9ad5-4c94-ba88-f134a4023b5e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:02.903310 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bca971-9ad5-4c94-ba88-f134a4023b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:02.903588 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bca971-9ad5-4c94-ba88-f134a4023b5e None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:02.904302 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bca971-9ad5-4c94-ba88-f134a4023b5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:02.904615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bca971-9ad5-4c94-ba88-f134a4023b5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:02.905990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bca971-9ad5-4c94-ba88-f134a4023b5e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:02.918325 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 425/843] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:02.923924 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8beea0-a6e4-4346-aeb4-33ea5422bf27 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:03.028932 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8beea0-a6e4-4346-aeb4-33ea5422bf27 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:03.063869 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed7e51-98bb-4985-8642-971b64c90f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:03.064668 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed7e51-98bb-4985-8642-971b64c90f82 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:03.065511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed7e51-98bb-4985-8642-971b64c90f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:03.065788 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed7e51-98bb-4985-8642-971b64c90f82 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:03.073531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed7e51-98bb-4985-8642-971b64c90f82 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:03.074974 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed7e51-98bb-4985-8642-971b64c90f82 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:03.089245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-2bed7e51-98bb-4985-8642-971b64c90f82 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 18:50:03.089245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:03.089245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:03.089245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:03.089245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:03.089245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:03.089245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:03.089245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:03.089245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:03.089245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:03.089245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:03.089245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:03.089245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:50:03.089245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:50:03.089245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:50:03.089245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:50:03.090160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:03.090160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:03.090160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:50:03.090160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:03.090160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:50:03.090160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:03.090160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:03.090160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:03.090160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:03.090160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:03.090160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:03.090160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:03.090160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:03.090160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:03.090160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:03.090160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:03.090160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:03.090160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:03.091081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:03.091081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:03.091081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:50:03.091081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:50:03.091081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:50:03.091081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:50:03.091081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:50:03.091081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:50:03.091081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 18:50:03.091081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:03.091081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 419/844] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:50:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:03.098141 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bca971-9ad5-4c94-ba88-f134a4023b5e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:03.201473 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bca971-9ad5-4c94-ba88-f134a4023b5e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:03.233250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:03.237500 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581b31f-02a1-4926-bfbc-b9b80476357f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:03.238184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581b31f-02a1-4926-bfbc-b9b80476357f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:03.238467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581b31f-02a1-4926-bfbc-b9b80476357f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:03.238741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581b31f-02a1-4926-bfbc-b9b80476357f None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:03.239599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d581b31f-02a1-4926-bfbc-b9b80476357f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:03.239909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d581b31f-02a1-4926-bfbc-b9b80476357f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:03.241341 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d581b31f-02a1-4926-bfbc-b9b80476357f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:03.261355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 426/845] 10.222.0.69 () {56 vars in 1135 bytes} [Thu Jul 14 18:50:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:03.265542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed7e51-98bb-4985-8642-971b64c90f82 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:03.292756 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:03.374388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed7e51-98bb-4985-8642-971b64c90f82 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:03.410893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82546bbe-3699-4bf4-b42a-064028e974c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:03.412029 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82546bbe-3699-4bf4-b42a-064028e974c6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:03.412318 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82546bbe-3699-4bf4-b42a-064028e974c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:03.412598 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82546bbe-3699-4bf4-b42a-064028e974c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:03.487911 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82546bbe-3699-4bf4-b42a-064028e974c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:03.489396 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82546bbe-3699-4bf4-b42a-064028e974c6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:03.517286 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:03.591987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 420/846] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:50:03 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:50:03.598371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581b31f-02a1-4926-bfbc-b9b80476357f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:03.635105 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0852d0-fc42-4655-b0c1-cbd5aa6edf47 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:03.635439 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0852d0-fc42-4655-b0c1-cbd5aa6edf47 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:03.635639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0852d0-fc42-4655-b0c1-cbd5aa6edf47 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:03.637299 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 421/847] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:50:03.642262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2918a-bacf-46b0-900b-57735105d18c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:03.642557 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2918a-bacf-46b0-900b-57735105d18c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:03.642846 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2918a-bacf-46b0-900b-57735105d18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:03.661378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-46b2918a-bacf-46b0-900b-57735105d18c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:03.662594 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-46b2918a-bacf-46b0-900b-57735105d18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:03.706241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581b31f-02a1-4926-bfbc-b9b80476357f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:03.740665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 422/848] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:03.743900 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6c91c-cee4-4ae2-9d73-20e31382ed73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:03.744936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6c91c-cee4-4ae2-9d73-20e31382ed73 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:03.745427 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6c91c-cee4-4ae2-9d73-20e31382ed73 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:03.745939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6c91c-cee4-4ae2-9d73-20e31382ed73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:03.757245 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6c91c-cee4-4ae2-9d73-20e31382ed73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:03.758726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2918a-bacf-46b0-900b-57735105d18c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:03.759353 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6c91c-cee4-4ae2-9d73-20e31382ed73 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:03.778859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-d5c6c91c-cee4-4ae2-9d73-20e31382ed73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:03.778859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:03.778859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:03.778859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:03.778859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:03.778859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:03.778859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:03.778859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:03.778859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:03.778859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:03.778859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:03.778859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:03.778859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:03.778859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:03.778859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:03.779974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:03.779974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:03.779974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:03.779974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:03.779974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:03.779974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:03.779974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:03.779974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:03.779974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:03.779974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:03.779974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:03.779974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:03.779974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:03.779974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:03.779974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:03.779974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:03.779974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:03.780868 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:03.780868 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:03.780868 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:03.780868 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:03.783077 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 427/849] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:03.788950 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6c91c-cee4-4ae2-9d73-20e31382ed73 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:03.813765 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:03.835991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:03.864368 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2918a-bacf-46b0-900b-57735105d18c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:03.902789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffef2c-09b7-435a-9cd5-f18bf863cc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:03.903414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6c91c-cee4-4ae2-9d73-20e31382ed73 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:03.903634 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffef2c-09b7-435a-9cd5-f18bf863cc13 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:03.903927 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffef2c-09b7-435a-9cd5-f18bf863cc13 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:03.904208 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffef2c-09b7-435a-9cd5-f18bf863cc13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:03.916257 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ffef2c-09b7-435a-9cd5-f18bf863cc13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:50:03.924734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-34ffef2c-09b7-435a-9cd5-f18bf863cc13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:50:03.924734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:03.924734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:03.924734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:03.924734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:03.924734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:03.924734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:03.924734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:03.924734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:03.924734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:03.924734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:03.924734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:03.924734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:50:03.924734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:50:03.924734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:50:03.924734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:50:03.925673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:50:03.925673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:50:03.925673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:50:03.925673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:50:03.925673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:03.925673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:03.925673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:03.925673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:03.925673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:03.925673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:03.925673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:03.925673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:03.925673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:03.925673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:03.925673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:03.925673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:03.926500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:03.926500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:03.926500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:03.926500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:03.926500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:50:03.926500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:50:03.926500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:50:03.926500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:50:03.926500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:50:03.926500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:50:03.926500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:50:03.926500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:03.930999 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 423/850] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:50:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:03.932444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffef2c-09b7-435a-9cd5-f18bf863cc13 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:03.951491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dbf15-5062-41f4-b3ab-5cb05c97333e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:03.952805 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dbf15-5062-41f4-b3ab-5cb05c97333e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:03.953688 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dbf15-5062-41f4-b3ab-5cb05c97333e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:03.954182 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dbf15-5062-41f4-b3ab-5cb05c97333e None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:03.955136 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958dbf15-5062-41f4-b3ab-5cb05c97333e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:03.955720 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958dbf15-5062-41f4-b3ab-5cb05c97333e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:03.958232 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958dbf15-5062-41f4-b3ab-5cb05c97333e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:03.979103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 428/851] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:03.981547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dbf15-5062-41f4-b3ab-5cb05c97333e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:04.053825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffef2c-09b7-435a-9cd5-f18bf863cc13 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:04.066818 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:04.088696 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8c0b7-1ffe-4120-84d1-beb91154782b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:04.089381 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8c0b7-1ffe-4120-84d1-beb91154782b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:04.089671 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8c0b7-1ffe-4120-84d1-beb91154782b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:04.089954 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8c0b7-1ffe-4120-84d1-beb91154782b None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:04.090810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8c0b7-1ffe-4120-84d1-beb91154782b None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:04.091133 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8c0b7-1ffe-4120-84d1-beb91154782b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:04.092586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8c0b7-1ffe-4120-84d1-beb91154782b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:04.103620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dbf15-5062-41f4-b3ab-5cb05c97333e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:04.107152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 424/852] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:50:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:04.108475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8c0b7-1ffe-4120-84d1-beb91154782b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:04.117270 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:04.143799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcaa53-6a28-4d4e-bd28-7b6378c4990d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:04.145175 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcaa53-6a28-4d4e-bd28-7b6378c4990d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:04.146550 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcaa53-6a28-4d4e-bd28-7b6378c4990d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:04.147020 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcaa53-6a28-4d4e-bd28-7b6378c4990d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:04.156217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcaa53-6a28-4d4e-bd28-7b6378c4990d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:04.158352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcaa53-6a28-4d4e-bd28-7b6378c4990d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:04.179379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c0dcaa53-6a28-4d4e-bd28-7b6378c4990d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:04.179379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:04.179379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:04.179379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:04.179379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:04.179379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:04.179379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:04.179379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:04.179379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:04.179379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:04.179379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:04.179379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:04.179379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:50:04.179379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:50:04.179379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:50:04.180330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:50:04.180330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:04.180330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:04.180330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:04.180330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:04.180330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:04.180330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:04.180330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:04.180330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:04.180330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:04.180330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:04.180330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:04.180330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:04.180330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:04.180330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:04.180330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:04.180330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:04.181242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:50:04.181242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:50:04.181242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:50:04.181242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:50:04.181242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:50:04.181242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:50:04.181242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:04.181242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:04.187997 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 429/853] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:50:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:04.189296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcaa53-6a28-4d4e-bd28-7b6378c4990d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:04.225563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8c0b7-1ffe-4120-84d1-beb91154782b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:04.264118 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb70a7-66a7-4791-8830-73450af60826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:04.265046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb70a7-66a7-4791-8830-73450af60826 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:04.265339 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb70a7-66a7-4791-8830-73450af60826 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:04.265636 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb70a7-66a7-4791-8830-73450af60826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:04.277093 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb70a7-66a7-4791-8830-73450af60826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:04.278665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb70a7-66a7-4791-8830-73450af60826 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:04.298823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-5ebb70a7-66a7-4791-8830-73450af60826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:04.298823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:04.298823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:04.298823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:04.298823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:04.298823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:04.298823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:04.298823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:04.298823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:04.298823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:04.298823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:04.298823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:04.298823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:04.298823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:04.298823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:04.300467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:04.300467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:04.300467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:04.300467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:04.300467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:04.300467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:04.300467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:04.300467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:04.300467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:04.300467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:04.300467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:04.300467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:04.300467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:04.300467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:04.300467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:04.300467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:04.300467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:04.301374 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:04.301374 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:04.301374 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:04.301374 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:04.301374 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 425/854] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:04.302362 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee78aa-00e6-4b41-be38-c29c8c1262ef None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:04.302468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee78aa-00e6-4b41-be38-c29c8c1262ef None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:04.302750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee78aa-00e6-4b41-be38-c29c8c1262ef None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:04.307276 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 426/855] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:04.307276 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee78aa-00e6-4b41-be38-c29c8c1262ef None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:04.310197 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcaa53-6a28-4d4e-bd28-7b6378c4990d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:04.335440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:04.344943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505b9fc-c3c2-4013-83e6-7fa99c8f43eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:04.345618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505b9fc-c3c2-4013-83e6-7fa99c8f43eb None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:04.345909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505b9fc-c3c2-4013-83e6-7fa99c8f43eb None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:04.346796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505b9fc-c3c2-4013-83e6-7fa99c8f43eb None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:04.347616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505b9fc-c3c2-4013-83e6-7fa99c8f43eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:04.347929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505b9fc-c3c2-4013-83e6-7fa99c8f43eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:04.349377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505b9fc-c3c2-4013-83e6-7fa99c8f43eb None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:04.361230 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 430/856] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:50:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:04.363060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06adc0-55ea-4456-810e-1083ec949f96 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:04.363425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06adc0-55ea-4456-810e-1083ec949f96 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:04.363855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06adc0-55ea-4456-810e-1083ec949f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:04.380375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3a06adc0-55ea-4456-810e-1083ec949f96 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:04.381585 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3a06adc0-55ea-4456-810e-1083ec949f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:04.392735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:04.425740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee78aa-00e6-4b41-be38-c29c8c1262ef None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:04.450574 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 431/857] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:04.451535 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06adc0-55ea-4456-810e-1083ec949f96 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:04.474262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2531ec-87cc-4131-897a-c672c4a5d53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:04.474944 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2531ec-87cc-4131-897a-c672c4a5d53d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:04.475272 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2531ec-87cc-4131-897a-c672c4a5d53d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:04.475556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2531ec-87cc-4131-897a-c672c4a5d53d None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:04.476365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2531ec-87cc-4131-897a-c672c4a5d53d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:04.476733 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2531ec-87cc-4131-897a-c672c4a5d53d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:04.478129 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2531ec-87cc-4131-897a-c672c4a5d53d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:04.492824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 427/858] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:04.497816 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2531ec-87cc-4131-897a-c672c4a5d53d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:04.575744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06adc0-55ea-4456-810e-1083ec949f96 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:04.588794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:04.604841 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2531ec-87cc-4131-897a-c672c4a5d53d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:04.611281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd51cdd-b9ff-4bb1-90ab-d076d3ae2a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:04.612480 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd51cdd-b9ff-4bb1-90ab-d076d3ae2a62 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:04.612762 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd51cdd-b9ff-4bb1-90ab-d076d3ae2a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:04.613113 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd51cdd-b9ff-4bb1-90ab-d076d3ae2a62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:04.613987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd51cdd-b9ff-4bb1-90ab-d076d3ae2a62 None admin] RBAC: Adding query filter params (role.id=27fc14631c494ca0b2dc7f708be016c2, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=4368f68724f946eca53084a9ec7ff590) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:04.614296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd51cdd-b9ff-4bb1-90ab-d076d3ae2a62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:04.615884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd51cdd-b9ff-4bb1-90ab-d076d3ae2a62 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:04.631209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 432/859] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:50:04 2022] GET /identity/v3/role_assignments?user.id=4368f68724f946eca53084a9ec7ff590&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:04.631209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd51cdd-b9ff-4bb1-90ab-d076d3ae2a62 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:04.643179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:04.657325 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4b722-b6d0-4f97-ba75-9fed4b9f299e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:04.658209 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4b722-b6d0-4f97-ba75-9fed4b9f299e None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:04.658940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4b722-b6d0-4f97-ba75-9fed4b9f299e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:04.659252 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4b722-b6d0-4f97-ba75-9fed4b9f299e None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:04.663420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f4b722-b6d0-4f97-ba75-9fed4b9f299e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:04.663420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f4b722-b6d0-4f97-ba75-9fed4b9f299e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:04.744948 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd51cdd-b9ff-4bb1-90ab-d076d3ae2a62 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:04.747369 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 428/860] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 14 18:50:04 2022] POST /identity/v3/services => generated 251 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:50:04.791887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626e2bb-7f73-4097-81ad-706a4c8ae567 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:04.792178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626e2bb-7f73-4097-81ad-706a4c8ae567 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:04.792462 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626e2bb-7f73-4097-81ad-706a4c8ae567 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:04.794050 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 429/861] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:50:04.799766 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b46251-2149-4260-89ea-a61e4317dc2c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:04.800426 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b46251-2149-4260-89ea-a61e4317dc2c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:04.800892 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b46251-2149-4260-89ea-a61e4317dc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:04.802017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b6c78-1c62-4d0b-afab-b5c4fb2a38ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:04.802865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b6c78-1c62-4d0b-afab-b5c4fb2a38ed None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:04.803156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b6c78-1c62-4d0b-afab-b5c4fb2a38ed None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:04.803464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b6c78-1c62-4d0b-afab-b5c4fb2a38ed None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:04.813217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b6c78-1c62-4d0b-afab-b5c4fb2a38ed None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:04.816207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b6c78-1c62-4d0b-afab-b5c4fb2a38ed None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:04.825078 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-17b46251-2149-4260-89ea-a61e4317dc2c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:04.826466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-17b46251-2149-4260-89ea-a61e4317dc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:04.839683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-610b6c78-1c62-4d0b-afab-b5c4fb2a38ed None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 18:50:04.839683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:04.839683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:04.839683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:04.839683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:04.839683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:04.839683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:04.839683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:04.839683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:04.839683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:04.839683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:04.839683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:04.839683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:50:04.839683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:50:04.839683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:50:04.839683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:50:04.840708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:04.840708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:04.840708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:50:04.840708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:04.840708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:50:04.840708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:04.840708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:04.840708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:04.840708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:04.840708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:04.840708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:04.840708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:04.840708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:04.840708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:04.840708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:04.840708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:04.840708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:04.840708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:04.841664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:04.841664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:04.841664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:50:04.841664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:50:04.841664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:50:04.841664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:50:04.841664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:50:04.841664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:50:04.841664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 18:50:04.841664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:04.841664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 433/862] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 14 18:50:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:04.855860 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b6c78-1c62-4d0b-afab-b5c4fb2a38ed None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:04.938862 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 430/863] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:04 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:04.947544 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:04.963667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b46251-2149-4260-89ea-a61e4317dc2c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:04.996921 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b6c78-1c62-4d0b-afab-b5c4fb2a38ed None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:05.043710 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73744cc3-10b2-43c7-a900-8e7a9276f19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:05.044398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73744cc3-10b2-43c7-a900-8e7a9276f19e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:05.044690 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73744cc3-10b2-43c7-a900-8e7a9276f19e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:05.044971 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73744cc3-10b2-43c7-a900-8e7a9276f19e None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:05.045823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73744cc3-10b2-43c7-a900-8e7a9276f19e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:05.046140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73744cc3-10b2-43c7-a900-8e7a9276f19e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:05.048181 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73744cc3-10b2-43c7-a900-8e7a9276f19e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:05.050010 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:05.077453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 434/864] 10.222.0.69 () {56 vars in 1149 bytes} [Thu Jul 14 18:50:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:05.077644 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b46251-2149-4260-89ea-a61e4317dc2c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:05.079772 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73744cc3-10b2-43c7-a900-8e7a9276f19e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:05.116999 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada9536-6184-4035-a47a-40003be2c792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:05.118691 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada9536-6184-4035-a47a-40003be2c792 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:05.119386 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada9536-6184-4035-a47a-40003be2c792 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:05.119890 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada9536-6184-4035-a47a-40003be2c792 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:05.121305 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada9536-6184-4035-a47a-40003be2c792 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:05.123116 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada9536-6184-4035-a47a-40003be2c792 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:05.140011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-3ada9536-6184-4035-a47a-40003be2c792 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 18:50:05.140011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:05.140011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:50:05.140011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:50:05.140011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:50:05.140011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:50:05.140011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:50:05.140011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:05.140011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 18:50:05.140011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:05.140011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:05.140011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:05.140011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:05.140011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:05.140011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:05.140011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:05.141006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:05.141006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:05.141006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:05.141006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:05.141006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:05.141006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:05.141006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:05.141006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:50:05.141006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:50:05.141006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:50:05.141006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:50:05.141006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:05.141006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:05.141006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:05.141006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:05.141006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:05.141831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:05.141831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:05.141831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:05.141831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:05.141831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:05.141831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:05.141831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:05.141831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:05.141831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:05.141831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:05.141831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:05.141831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:50:05.141831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:05.141831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 18:50:05.141831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:05.145080 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 431/865] 10.222.0.69 () {56 vars in 1132 bytes} [Thu Jul 14 18:50:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:05.151556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada9536-6184-4035-a47a-40003be2c792 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:05.199923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73744cc3-10b2-43c7-a900-8e7a9276f19e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:05.227601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:05.236007 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de1806-4d28-4717-813f-7573722df918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:05.237472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de1806-4d28-4717-813f-7573722df918 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:05.238032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de1806-4d28-4717-813f-7573722df918 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:05.238532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de1806-4d28-4717-813f-7573722df918 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:05.259822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada9536-6184-4035-a47a-40003be2c792 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:05.297341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6c401-d056-43f3-8128-0059d853d81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:05.298037 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6c401-d056-43f3-8128-0059d853d81b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:05.298356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6c401-d056-43f3-8128-0059d853d81b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:05.298636 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6c401-d056-43f3-8128-0059d853d81b None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:05.300014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6c401-d056-43f3-8128-0059d853d81b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:05.300334 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6c401-d056-43f3-8128-0059d853d81b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:05.301817 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6c401-d056-43f3-8128-0059d853d81b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:05.311734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 432/866] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 14 18:50:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:05.313018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6c401-d056-43f3-8128-0059d853d81b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:05.319865 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:05.323929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de1806-4d28-4717-813f-7573722df918 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:05.325409 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de1806-4d28-4717-813f-7573722df918 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:05.454077 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 435/867] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:50:05 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:50:05.455058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de1806-4d28-4717-813f-7573722df918 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:05.466405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6c401-d056-43f3-8128-0059d853d81b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:05.483350 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:05.515601 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81d978-9ddd-452e-9d3c-d1a98d7158d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:05.517022 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81d978-9ddd-452e-9d3c-d1a98d7158d7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:05.517674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81d978-9ddd-452e-9d3c-d1a98d7158d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:05.518183 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81d978-9ddd-452e-9d3c-d1a98d7158d7 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:05.519701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81d978-9ddd-452e-9d3c-d1a98d7158d7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:05.520240 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81d978-9ddd-452e-9d3c-d1a98d7158d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:05.522621 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81d978-9ddd-452e-9d3c-d1a98d7158d7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:05.535720 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 433/868] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 14 18:50:05 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:05.574406 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de1806-4d28-4717-813f-7573722df918 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:05.613425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf06cf1-f9d0-4943-8af9-901eed66bcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:05.614289 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf06cf1-f9d0-4943-8af9-901eed66bcbf None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:05.614584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf06cf1-f9d0-4943-8af9-901eed66bcbf None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:05.615005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf06cf1-f9d0-4943-8af9-901eed66bcbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:05.626694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf06cf1-f9d0-4943-8af9-901eed66bcbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:05.628340 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf06cf1-f9d0-4943-8af9-901eed66bcbf None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:05.647892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-2cf06cf1-f9d0-4943-8af9-901eed66bcbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:05.647892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:05.647892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:05.647892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:05.647892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:05.647892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:05.647892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:05.647892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:05.647892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:05.647892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:05.647892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:05.647892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:05.647892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:05.647892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:05.647892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:05.648856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:05.648856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:05.648856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:05.648856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:05.648856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:05.648856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:05.648856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:05.648856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:05.648856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:05.648856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:05.648856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:05.648856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:05.648856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:05.648856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:05.648856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:05.648856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:05.648856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:05.649736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:05.649736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:05.649736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:05.649736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:05.649736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 436/869] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:05.660393 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81d978-9ddd-452e-9d3c-d1a98d7158d7 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:05.738449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:05.775298 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81d978-9ddd-452e-9d3c-d1a98d7158d7 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:05.822060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3776d-7e4c-486d-bdf4-da47a47e365b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:05.822797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3776d-7e4c-486d-bdf4-da47a47e365b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:05.823085 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3776d-7e4c-486d-bdf4-da47a47e365b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:05.823397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3776d-7e4c-486d-bdf4-da47a47e365b None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:05.824143 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3776d-7e4c-486d-bdf4-da47a47e365b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:05.824456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3776d-7e4c-486d-bdf4-da47a47e365b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:05.825883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3776d-7e4c-486d-bdf4-da47a47e365b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:05.840674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 434/870] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:05.846702 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf06cf1-f9d0-4943-8af9-901eed66bcbf None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:05.931991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:05.956620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf06cf1-f9d0-4943-8af9-901eed66bcbf None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:05.991966 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909076a-2133-446c-862c-e24ba279823f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:05.992695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909076a-2133-446c-862c-e24ba279823f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:05.992985 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909076a-2133-446c-862c-e24ba279823f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:05.993265 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909076a-2133-446c-862c-e24ba279823f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:06.001618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909076a-2133-446c-862c-e24ba279823f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:06.003275 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909076a-2133-446c-862c-e24ba279823f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:06.017548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c909076a-2133-446c-862c-e24ba279823f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:06.017548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:06.017548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:06.017548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:06.017548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:06.017548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:06.017548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:06.017548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:06.017548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:06.017548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:06.017548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:06.017548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:06.017548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:50:06.017548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:50:06.017548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:50:06.018991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:50:06.018991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:06.018991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:06.018991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:06.018991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:06.018991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:06.018991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:06.018991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:06.018991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:06.018991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:06.018991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:06.018991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:06.018991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:06.018991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:06.018991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:06.018991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:06.018991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:06.019942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:50:06.019942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:50:06.019942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:50:06.019942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:50:06.019942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:50:06.019942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:50:06.019942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:06.019942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:06.019942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 437/871] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:50:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:06.025501 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3776d-7e4c-486d-bdf4-da47a47e365b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:06.130868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3776d-7e4c-486d-bdf4-da47a47e365b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:06.159694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:06.168217 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067d643-8549-4d00-84fb-74a532cfec52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:06.168885 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067d643-8549-4d00-84fb-74a532cfec52 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:06.169172 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067d643-8549-4d00-84fb-74a532cfec52 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:06.169518 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067d643-8549-4d00-84fb-74a532cfec52 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:06.170428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067d643-8549-4d00-84fb-74a532cfec52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:06.170745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067d643-8549-4d00-84fb-74a532cfec52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:06.172205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067d643-8549-4d00-84fb-74a532cfec52 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:06.184664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 435/872] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:50:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:06.190501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909076a-2133-446c-862c-e24ba279823f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:06.298711 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909076a-2133-446c-862c-e24ba279823f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:06.334259 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a38857e-fa7c-465a-9ad2-ac880516d305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:06.334937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a38857e-fa7c-465a-9ad2-ac880516d305 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:06.335252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a38857e-fa7c-465a-9ad2-ac880516d305 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:06.335539 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a38857e-fa7c-465a-9ad2-ac880516d305 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:06.336368 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a38857e-fa7c-465a-9ad2-ac880516d305 None admin] RBAC: Adding query filter params (role.id=24846e82841b4b05b4d9e9d193a98868, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=4ef0fad9a71d43bfbaba9c87dfed41c4) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:06.336684 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a38857e-fa7c-465a-9ad2-ac880516d305 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:06.338127 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a38857e-fa7c-465a-9ad2-ac880516d305 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:06.344576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:06.344818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 438/873] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:50:06 2022] GET /identity/v3/role_assignments?user.id=4ef0fad9a71d43bfbaba9c87dfed41c4&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=24846e82841b4b05b4d9e9d193a98868 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:06.740573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3166943-d778-4860-b684-16443f08a185 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:06.741280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3166943-d778-4860-b684-16443f08a185 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:06.741280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3166943-d778-4860-b684-16443f08a185 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:06.742742 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 436/874] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:06.747953 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f2fb0-cdde-4826-9fb5-d35c171de071 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:06.748434 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f2fb0-cdde-4826-9fb5-d35c171de071 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:06.748603 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f2fb0-cdde-4826-9fb5-d35c171de071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:06.765879 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6e3f2fb0-cdde-4826-9fb5-d35c171de071 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:06.767061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6e3f2fb0-cdde-4826-9fb5-d35c171de071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:06.835662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 439/875] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:06 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:06.857745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3166943-d778-4860-b684-16443f08a185 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:06.901544 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d518f3e-b470-4146-bf9a-d6bacbb6f09d None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:06.902112 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d518f3e-b470-4146-bf9a-d6bacbb6f09d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:06.902358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d518f3e-b470-4146-bf9a-d6bacbb6f09d None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:06.904019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 440/876] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:06.909441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b16a9-f0fc-4856-8962-1c2948a0e871 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:06.909721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b16a9-f0fc-4856-8962-1c2948a0e871 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:06.910000 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b16a9-f0fc-4856-8962-1c2948a0e871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:06.926471 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9d4b16a9-f0fc-4856-8962-1c2948a0e871 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:06.927746 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9d4b16a9-f0fc-4856-8962-1c2948a0e871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:06.987407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3166943-d778-4860-b684-16443f08a185 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:07.000591 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 441/877] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:06 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:07.027660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b16a9-f0fc-4856-8962-1c2948a0e871 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:07.035494 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea101a1d-5f04-42b9-b88e-9b665c97e9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:07.036647 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea101a1d-5f04-42b9-b88e-9b665c97e9ef None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:07.037163 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea101a1d-5f04-42b9-b88e-9b665c97e9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:07.037658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea101a1d-5f04-42b9-b88e-9b665c97e9ef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:07.038594 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea101a1d-5f04-42b9-b88e-9b665c97e9ef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:07.040260 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea101a1d-5f04-42b9-b88e-9b665c97e9ef None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:07.053498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-ea101a1d-5f04-42b9-b88e-9b665c97e9ef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 18:50:07.053498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:07.053498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:50:07.053498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:50:07.053498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:50:07.053498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:50:07.053498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:50:07.053498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:07.053498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 18:50:07.053498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:07.053498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:07.053498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:07.053498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:07.053498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:07.053498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:07.053498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:07.053498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:07.054535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:07.054535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:07.054535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:07.054535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:07.054535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:07.054535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:07.054535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:50:07.054535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:50:07.054535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:50:07.054535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:50:07.054535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:07.054535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:07.054535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:07.054535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:07.054535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:07.054535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:07.055365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:07.055365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:07.055365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:07.055365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:07.055365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:07.055365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:07.055365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:07.055365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:07.055365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:07.055365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:07.055365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:50:07.055365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:07.055365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 18:50:07.055365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:07.062809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 437/878] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 14 18:50:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:07.063909 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea101a1d-5f04-42b9-b88e-9b665c97e9ef None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:07.092098 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:07.120408 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:07.137695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b16a9-f0fc-4856-8962-1c2948a0e871 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:07.170267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea101a1d-5f04-42b9-b88e-9b665c97e9ef None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:07.174621 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd8e66-49f6-40ad-bb4a-feb6abb198a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:07.175477 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd8e66-49f6-40ad-bb4a-feb6abb198a7 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:07.175764 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd8e66-49f6-40ad-bb4a-feb6abb198a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:07.176039 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd8e66-49f6-40ad-bb4a-feb6abb198a7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:07.176698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd8e66-49f6-40ad-bb4a-feb6abb198a7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:07.178092 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd8e66-49f6-40ad-bb4a-feb6abb198a7 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:07.193126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-d9bd8e66-49f6-40ad-bb4a-feb6abb198a7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 18:50:07.193126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:07.193126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:50:07.193126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:50:07.193126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:50:07.193126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:50:07.193126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:50:07.193126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:07.193126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 18:50:07.193126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:07.193126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:07.193126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:07.193126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:07.193126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:07.193126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:07.193126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:07.193126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:07.194179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:07.194179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:07.194179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:07.194179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:07.194179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:07.194179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:07.194179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:50:07.194179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:50:07.194179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:50:07.194179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:50:07.194179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:07.194179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:07.194179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:07.194179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:07.194179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:07.194179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:07.195017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:07.195017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:07.195017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:07.195017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:07.195017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:07.195017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:07.195017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:07.195017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:07.195017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:07.195017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:07.195017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:50:07.195017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:07.195017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 18:50:07.195017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:07.199297 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 442/879] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:50:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:07.200135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd8e66-49f6-40ad-bb4a-feb6abb198a7 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:07.213090 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676d25f-d972-4522-b447-b7bb7e166511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:07.214090 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676d25f-d972-4522-b447-b7bb7e166511 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:07.214575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676d25f-d972-4522-b447-b7bb7e166511 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:07.215070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676d25f-d972-4522-b447-b7bb7e166511 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:07.215999 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676d25f-d972-4522-b447-b7bb7e166511 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:07.216503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676d25f-d972-4522-b447-b7bb7e166511 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:07.218106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676d25f-d972-4522-b447-b7bb7e166511 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:07.233005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 438/880] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 14 18:50:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:07.234209 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676d25f-d972-4522-b447-b7bb7e166511 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:07.309113 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd8e66-49f6-40ad-bb4a-feb6abb198a7 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:07.335349 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:07.338766 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676d25f-d972-4522-b447-b7bb7e166511 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:07.347157 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c744b-d495-47ff-9062-7f8ff96bf809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:07.351055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c744b-d495-47ff-9062-7f8ff96bf809 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:07.351055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c744b-d495-47ff-9062-7f8ff96bf809 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:07.351055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c744b-d495-47ff-9062-7f8ff96bf809 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:07.351055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c744b-d495-47ff-9062-7f8ff96bf809 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:07.351055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c744b-d495-47ff-9062-7f8ff96bf809 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:07.351055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c744b-d495-47ff-9062-7f8ff96bf809 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:07.360898 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:07.361186 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 443/881] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:50:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:07.362666 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c744b-d495-47ff-9062-7f8ff96bf809 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:07.376115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25f733-73ee-4a7f-a1d7-1f589d0e1bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:07.377152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25f733-73ee-4a7f-a1d7-1f589d0e1bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:07.378201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25f733-73ee-4a7f-a1d7-1f589d0e1bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:07.378673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25f733-73ee-4a7f-a1d7-1f589d0e1bb0 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:07.379604 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25f733-73ee-4a7f-a1d7-1f589d0e1bb0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:07.380191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25f733-73ee-4a7f-a1d7-1f589d0e1bb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:07.381895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25f733-73ee-4a7f-a1d7-1f589d0e1bb0 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:07.395482 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 439/882] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 14 18:50:07 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:07.479718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c744b-d495-47ff-9062-7f8ff96bf809 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:07.519755 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7909a-9bcd-4baf-bbb9-16707e3dd274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:07.519960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7909a-9bcd-4baf-bbb9-16707e3dd274 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:07.520196 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7909a-9bcd-4baf-bbb9-16707e3dd274 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:07.520395 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7909a-9bcd-4baf-bbb9-16707e3dd274 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:07.521115 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7909a-9bcd-4baf-bbb9-16707e3dd274 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:07.521420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7909a-9bcd-4baf-bbb9-16707e3dd274 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:07.522901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7909a-9bcd-4baf-bbb9-16707e3dd274 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:07.529139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 444/883] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:50:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:07.535359 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25f733-73ee-4a7f-a1d7-1f589d0e1bb0 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:07.634369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8358b-3653-461e-9212-e5e2b8c15cdb None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:07.634700 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8358b-3653-461e-9212-e5e2b8c15cdb None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:07.634936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8358b-3653-461e-9212-e5e2b8c15cdb None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:07.636595 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 445/884] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:07.641543 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f8113-66c7-4573-8f48-1e0187c2f11f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:07.641833 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f8113-66c7-4573-8f48-1e0187c2f11f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:07.642125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f8113-66c7-4573-8f48-1e0187c2f11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:07.649759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25f733-73ee-4a7f-a1d7-1f589d0e1bb0 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:07.659414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ff8f8113-66c7-4573-8f48-1e0187c2f11f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:07.660610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ff8f8113-66c7-4573-8f48-1e0187c2f11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:07.685480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aea6cd-b1e0-4b35-9c4e-6b18926504fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:07.686150 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aea6cd-b1e0-4b35-9c4e-6b18926504fe None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:07.686443 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aea6cd-b1e0-4b35-9c4e-6b18926504fe None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:07.686727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aea6cd-b1e0-4b35-9c4e-6b18926504fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:07.687656 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aea6cd-b1e0-4b35-9c4e-6b18926504fe None admin] RBAC: Adding query filter params (interface=public, service_id=ed5dd95d7b364feeb7159d238b721cb1, region_id=RegionOne) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:07.687978 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aea6cd-b1e0-4b35-9c4e-6b18926504fe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:07.689456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aea6cd-b1e0-4b35-9c4e-6b18926504fe None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:07.695831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 440/885] 10.222.0.69 () {56 vars in 1251 bytes} [Thu Jul 14 18:50:07 2022] GET /identity/v3/endpoints?service_id=ed5dd95d7b364feeb7159d238b721cb1&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:07.701701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aea6cd-b1e0-4b35-9c4e-6b18926504fe None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:07.718524 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:07.751188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 446/886] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:07 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:07.769103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:07.769990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f8113-66c7-4573-8f48-1e0187c2f11f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:07.831687 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aea6cd-b1e0-4b35-9c4e-6b18926504fe None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:07.880561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a25ef0-e8c5-49f0-bdd0-57607a6d089a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:07.881656 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a25ef0-e8c5-49f0-bdd0-57607a6d089a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:07.882146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a25ef0-e8c5-49f0-bdd0-57607a6d089a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:07.882626 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a25ef0-e8c5-49f0-bdd0-57607a6d089a None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:07.884043 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a25ef0-e8c5-49f0-bdd0-57607a6d089a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:07.886222 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a25ef0-e8c5-49f0-bdd0-57607a6d089a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:07.903273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f8113-66c7-4573-8f48-1e0187c2f11f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:07.903475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 441/887] 10.222.0.69 () {56 vars in 1087 bytes} [Thu Jul 14 18:50:07 2022] GET /identity/v3/services => generated 1011 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:50:07.945464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce80b0b-6d6b-429c-9bf9-f667268cf598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:07.946211 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce80b0b-6d6b-429c-9bf9-f667268cf598 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:07.946505 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce80b0b-6d6b-429c-9bf9-f667268cf598 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:07.946786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce80b0b-6d6b-429c-9bf9-f667268cf598 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:07.947622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce80b0b-6d6b-429c-9bf9-f667268cf598 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:07.949020 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce80b0b-6d6b-429c-9bf9-f667268cf598 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:08.024398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 447/888] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 14 18:50:07 2022] POST /identity/v3/services => generated 269 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:50:08.484483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4856ff-4ea7-41fb-ba14-c8ba2c206222 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:08.484865 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4856ff-4ea7-41fb-ba14-c8ba2c206222 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:08.485165 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4856ff-4ea7-41fb-ba14-c8ba2c206222 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:08.486645 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 442/889] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:08.492147 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f63f4d-0a92-4eb7-adcf-de8e1ec0bde7 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:08.492698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f63f4d-0a92-4eb7-adcf-de8e1ec0bde7 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:08.493186 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f63f4d-0a92-4eb7-adcf-de8e1ec0bde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:08.510747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-37f63f4d-0a92-4eb7-adcf-de8e1ec0bde7 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:08.512275 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-37f63f4d-0a92-4eb7-adcf-de8e1ec0bde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:08.558993 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:08.601516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 448/890] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:08.628807 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4856ff-4ea7-41fb-ba14-c8ba2c206222 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:08.741320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4856ff-4ea7-41fb-ba14-c8ba2c206222 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:08.751155 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:08.778857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7736033-1fae-43da-99b2-c5775fe3583b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:08.779713 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7736033-1fae-43da-99b2-c5775fe3583b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:08.781283 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7736033-1fae-43da-99b2-c5775fe3583b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:08.781283 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7736033-1fae-43da-99b2-c5775fe3583b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:08.791883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7736033-1fae-43da-99b2-c5775fe3583b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:50:08.800245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b7736033-1fae-43da-99b2-c5775fe3583b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 18:50:08.800245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:08.800245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:08.800245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:08.800245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:08.800245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:08.800245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:08.800245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:08.800245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:08.800245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:08.800245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:08.800245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:08.800245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:50:08.800245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:50:08.800245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:50:08.800245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:50:08.801185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:50:08.801185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:50:08.801185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:50:08.801185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:50:08.801185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:08.801185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:08.801185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:08.801185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:08.801185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:08.801185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:08.801185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:08.801185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:08.801185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:08.801185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:08.801185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:08.801185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:08.802409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:08.802409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:08.802409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:08.802409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:08.802409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:50:08.802409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:50:08.802409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:50:08.802409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:50:08.802409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:50:08.802409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:50:08.802409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 18:50:08.802409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:08.802409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 443/891] 10.222.0.69 () {56 vars in 1120 bytes} [Thu Jul 14 18:50:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:08.809415 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f63f4d-0a92-4eb7-adcf-de8e1ec0bde7 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:08.961155 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f63f4d-0a92-4eb7-adcf-de8e1ec0bde7 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:08.998134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226a85d-249b-40df-814f-d9e01c68acb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:08.998739 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226a85d-249b-40df-814f-d9e01c68acb5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:08.999029 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226a85d-249b-40df-814f-d9e01c68acb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:08.999333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226a85d-249b-40df-814f-d9e01c68acb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:09.000087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226a85d-249b-40df-814f-d9e01c68acb5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:09.000398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226a85d-249b-40df-814f-d9e01c68acb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:09.001904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226a85d-249b-40df-814f-d9e01c68acb5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:09.011989 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 449/892] 10.222.0.69 () {56 vars in 1115 bytes} [Thu Jul 14 18:50:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:09.020378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7736033-1fae-43da-99b2-c5775fe3583b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:09.207591 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7736033-1fae-43da-99b2-c5775fe3583b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:09.258730 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ee8b7-11f6-4876-8522-ab90f53e2b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:09.260086 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ee8b7-11f6-4876-8522-ab90f53e2b47 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:09.260793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ee8b7-11f6-4876-8522-ab90f53e2b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:09.261450 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ee8b7-11f6-4876-8522-ab90f53e2b47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:09.276359 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ee8b7-11f6-4876-8522-ab90f53e2b47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:09.278107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ee8b7-11f6-4876-8522-ab90f53e2b47 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:09.296542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-075ee8b7-11f6-4876-8522-ab90f53e2b47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:09.296542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:09.296542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:09.296542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:09.296542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:09.296542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:09.296542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:09.296542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:09.296542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:09.296542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:09.296542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:09.296542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:09.296542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:09.296542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:09.296542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:09.297456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:09.297456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:09.297456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:09.297456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:09.297456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:09.297456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:09.297456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:09.297456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:09.297456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:09.297456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:09.297456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:09.297456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:09.297456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:09.297456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:09.297456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:09.297456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:09.297456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:09.298766 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:09.298766 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:09.298766 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:09.298766 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:09.298766 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 444/893] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:09.304674 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226a85d-249b-40df-814f-d9e01c68acb5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:09.336931 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:09.410178 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226a85d-249b-40df-814f-d9e01c68acb5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:09.448954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3ab24-92a7-4a13-8852-d3c974668045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:09.449930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3ab24-92a7-4a13-8852-d3c974668045 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:09.450364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3ab24-92a7-4a13-8852-d3c974668045 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:09.450806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3ab24-92a7-4a13-8852-d3c974668045 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:09.452708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3ab24-92a7-4a13-8852-d3c974668045 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:09.453364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3ab24-92a7-4a13-8852-d3c974668045 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:09.454906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3ab24-92a7-4a13-8852-d3c974668045 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:09.469970 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 450/894] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:09.476252 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ee8b7-11f6-4876-8522-ab90f53e2b47 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:09.506963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:09.600396 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ee8b7-11f6-4876-8522-ab90f53e2b47 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:09.601569 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165884b-43d3-46ec-b8d8-7432b851adbc None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:09.602511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165884b-43d3-46ec-b8d8-7432b851adbc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:09.603875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165884b-43d3-46ec-b8d8-7432b851adbc None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:09.605795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 451/895] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:09.616734 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b65272-2f3c-418f-9f35-28e4b90a226a None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:09.617376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b65272-2f3c-418f-9f35-28e4b90a226a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:09.617881 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b65272-2f3c-418f-9f35-28e4b90a226a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:09.647994 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-08b65272-2f3c-418f-9f35-28e4b90a226a None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:09.649330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-08b65272-2f3c-418f-9f35-28e4b90a226a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:09.658975 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d131a4-bc04-4bc7-8eef-71e68f068481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:09.660198 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d131a4-bc04-4bc7-8eef-71e68f068481 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:09.660799 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d131a4-bc04-4bc7-8eef-71e68f068481 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:09.661279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d131a4-bc04-4bc7-8eef-71e68f068481 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:09.669825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d131a4-bc04-4bc7-8eef-71e68f068481 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:09.671635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d131a4-bc04-4bc7-8eef-71e68f068481 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:09.690065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b7d131a4-bc04-4bc7-8eef-71e68f068481 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 18:50:09.690065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:09.690065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:09.690065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:09.690065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:09.690065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:09.690065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:09.690065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:09.690065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:09.690065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:09.690065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:09.690065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:09.690065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:50:09.690065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:50:09.690065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:50:09.690065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:50:09.691049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:09.691049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:09.691049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:50:09.691049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:09.691049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:50:09.691049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:09.691049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:09.691049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:09.691049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:09.691049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:09.691049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:09.691049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:09.691049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:09.691049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:09.691049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:09.691049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:09.691049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:09.691049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:09.692015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:09.692015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:09.692015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:50:09.692015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:50:09.692015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:50:09.692015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:50:09.692015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:50:09.692015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:50:09.692015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 18:50:09.692015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:09.694737 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 445/896] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 14 18:50:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:09.700389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d131a4-bc04-4bc7-8eef-71e68f068481 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:09.731537 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 452/897] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:09 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:09.749444 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b65272-2f3c-418f-9f35-28e4b90a226a None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:09.774306 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:09.788164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:09.808227 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d131a4-bc04-4bc7-8eef-71e68f068481 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:09.843652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ada2be-8fb9-4144-bc80-289e9b6ee057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:09.844650 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ada2be-8fb9-4144-bc80-289e9b6ee057 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:09.845081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ada2be-8fb9-4144-bc80-289e9b6ee057 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:09.845504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ada2be-8fb9-4144-bc80-289e9b6ee057 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:09.847316 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ada2be-8fb9-4144-bc80-289e9b6ee057 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:09.847798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ada2be-8fb9-4144-bc80-289e9b6ee057 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:09.849366 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ada2be-8fb9-4144-bc80-289e9b6ee057 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:09.860019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b65272-2f3c-418f-9f35-28e4b90a226a None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:09.873450 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 446/898] 10.222.0.69 () {56 vars in 1149 bytes} [Thu Jul 14 18:50:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:09.874588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ada2be-8fb9-4144-bc80-289e9b6ee057 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:09.900497 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43c8d3-ae63-452e-a866-e943118f6add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:09.901158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43c8d3-ae63-452e-a866-e943118f6add None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:09.901450 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43c8d3-ae63-452e-a866-e943118f6add None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:09.901985 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43c8d3-ae63-452e-a866-e943118f6add None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:09.902813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc43c8d3-ae63-452e-a866-e943118f6add None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:09.904342 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc43c8d3-ae63-452e-a866-e943118f6add None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:09.986872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 453/899] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 14 18:50:09 2022] POST /identity/v3/services => generated 241 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:50:09.990464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ada2be-8fb9-4144-bc80-289e9b6ee057 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:10.038685 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6595159-76d5-4b10-a28b-a880642ba9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:10.039395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6595159-76d5-4b10-a28b-a880642ba9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:10.039684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6595159-76d5-4b10-a28b-a880642ba9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:10.039967 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6595159-76d5-4b10-a28b-a880642ba9f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:10.064462 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:10.117710 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6595159-76d5-4b10-a28b-a880642ba9f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:10.119110 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6595159-76d5-4b10-a28b-a880642ba9f0 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:10.121542 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbbaa7-de20-4606-bf8c-e06fc4a7dae8 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:10.121834 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbbaa7-de20-4606-bf8c-e06fc4a7dae8 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:10.122118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbbaa7-de20-4606-bf8c-e06fc4a7dae8 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:10.127783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 454/900] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:50:10.130042 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c17639-18a3-4bc2-94b0-9f4e444ccab6 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:10.130335 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c17639-18a3-4bc2-94b0-9f4e444ccab6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:10.130619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c17639-18a3-4bc2-94b0-9f4e444ccab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:10.149364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-f6c17639-18a3-4bc2-94b0-9f4e444ccab6 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:10.150768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f6c17639-18a3-4bc2-94b0-9f4e444ccab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:10.227552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 455/901] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:10.228703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cff750-4a87-450a-9059-934ae882e731 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:10.228999 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cff750-4a87-450a-9059-934ae882e731 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:10.229277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cff750-4a87-450a-9059-934ae882e731 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:10.230679 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 456/902] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:10.235824 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9997c-503c-4695-9709-79e126959d41 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:10.236119 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9997c-503c-4695-9709-79e126959d41 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:10.236407 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9997c-503c-4695-9709-79e126959d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:10.239205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 447/903] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:50:09 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:50:10.243290 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6595159-76d5-4b10-a28b-a880642ba9f0 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:10.255884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3db9997c-503c-4695-9709-79e126959d41 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:10.257252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3db9997c-503c-4695-9709-79e126959d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:10.261373 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:10.285054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:10.337151 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 457/904] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:10.338409 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9997c-503c-4695-9709-79e126959d41 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:10.360343 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6595159-76d5-4b10-a28b-a880642ba9f0 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:10.399609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3b4fd-2f3b-4819-8a18-a11110217fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:10.400783 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3b4fd-2f3b-4819-8a18-a11110217fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:10.401257 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3b4fd-2f3b-4819-8a18-a11110217fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:10.401975 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3b4fd-2f3b-4819-8a18-a11110217fb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:10.413375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea3b4fd-2f3b-4819-8a18-a11110217fb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:10.415198 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea3b4fd-2f3b-4819-8a18-a11110217fb7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:10.435820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-bea3b4fd-2f3b-4819-8a18-a11110217fb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:10.435820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:10.435820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:10.435820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:10.435820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:10.435820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:10.435820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:10.435820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:10.435820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:10.435820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:10.435820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:10.435820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:10.435820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:10.435820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:10.435820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:10.436976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:10.436976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:10.436976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:10.436976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:10.436976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:10.436976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:10.436976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:10.436976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:10.436976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:10.436976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:10.436976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:10.436976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:10.436976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:10.436976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:10.436976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:10.436976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:10.436976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:10.438084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:10.438084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:10.438084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:10.438084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:10.444756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9997c-503c-4695-9709-79e126959d41 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:10.444911 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 448/905] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:10.444911 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3b4fd-2f3b-4819-8a18-a11110217fb7 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:10.479456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5379d-d828-4359-8ac5-eb3068d6ed5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:10.480362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5379d-d828-4359-8ac5-eb3068d6ed5d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:10.480655 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5379d-d828-4359-8ac5-eb3068d6ed5d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:10.480934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5379d-d828-4359-8ac5-eb3068d6ed5d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:10.481614 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5379d-d828-4359-8ac5-eb3068d6ed5d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:10.482966 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5379d-d828-4359-8ac5-eb3068d6ed5d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:10.495629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-b1e5379d-d828-4359-8ac5-eb3068d6ed5d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 18:50:10.495629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:10.495629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:50:10.495629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:50:10.495629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:50:10.495629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:50:10.495629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:50:10.495629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:10.495629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 18:50:10.495629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:10.495629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:10.495629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:10.495629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:10.495629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:10.495629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:10.495629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:10.495629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:10.496743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:10.496743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:10.496743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:10.496743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:10.496743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:10.496743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:10.496743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:50:10.496743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:50:10.496743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:50:10.496743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:50:10.496743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:10.496743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:10.496743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:10.496743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:10.496743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:10.496743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:10.497573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:10.497573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:10.497573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:10.497573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:10.497573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:10.497573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:10.497573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:10.497573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:10.497573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:10.497573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:10.497573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:50:10.497573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:10.497573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 18:50:10.497573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:10.499262 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 458/906] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:50:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:10.500067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5379d-d828-4359-8ac5-eb3068d6ed5d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:10.506588 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:10.533393 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:10.550545 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3b4fd-2f3b-4819-8a18-a11110217fb7 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:10.590187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fd302-81f6-4320-9170-c397c73ef114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:10.591018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fd302-81f6-4320-9170-c397c73ef114 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:10.591341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fd302-81f6-4320-9170-c397c73ef114 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:10.591626 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fd302-81f6-4320-9170-c397c73ef114 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:10.592281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471fd302-81f6-4320-9170-c397c73ef114 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:10.593670 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471fd302-81f6-4320-9170-c397c73ef114 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:10.607677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-471fd302-81f6-4320-9170-c397c73ef114 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 18:50:10.607677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:10.607677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:50:10.607677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:50:10.607677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:50:10.607677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:50:10.607677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:50:10.607677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:10.607677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 18:50:10.607677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:10.607677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:10.607677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:10.607677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:10.607677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:10.607677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:10.607677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:10.608652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:10.608652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:10.608652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:10.608652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:10.608652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:10.608652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:10.608652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:10.608652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:50:10.608652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:50:10.608652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:50:10.608652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:50:10.608652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:10.608652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:10.608652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:10.608652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:10.608652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:10.615250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:10.615250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:10.615250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:10.615250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:10.615250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:10.615250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:10.615250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:10.615250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:10.615250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:10.615250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:10.615250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:10.615250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:50:10.615250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:10.615250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 18:50:10.615250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:10.616082 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5379d-d828-4359-8ac5-eb3068d6ed5d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:10.616159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 449/907] 10.222.0.69 () {56 vars in 1132 bytes} [Thu Jul 14 18:50:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:10.616159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fd302-81f6-4320-9170-c397c73ef114 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:10.648771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b925af3-519d-46f4-96b0-9e799f9c838a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:10.649442 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b925af3-519d-46f4-96b0-9e799f9c838a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:10.649784 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b925af3-519d-46f4-96b0-9e799f9c838a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:10.650002 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b925af3-519d-46f4-96b0-9e799f9c838a None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:10.650717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b925af3-519d-46f4-96b0-9e799f9c838a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:10.651027 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b925af3-519d-46f4-96b0-9e799f9c838a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:10.652701 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b925af3-519d-46f4-96b0-9e799f9c838a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:10.666698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 459/908] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:10.667538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b925af3-519d-46f4-96b0-9e799f9c838a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:10.722363 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fd302-81f6-4320-9170-c397c73ef114 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:10.733386 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:10.759324 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0f413-1291-4720-bcb5-ec57d7decea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:10.760093 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0f413-1291-4720-bcb5-ec57d7decea0 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:10.760418 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0f413-1291-4720-bcb5-ec57d7decea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:10.760966 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0f413-1291-4720-bcb5-ec57d7decea0 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:10.761450 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0f413-1291-4720-bcb5-ec57d7decea0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:10.761763 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0f413-1291-4720-bcb5-ec57d7decea0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:10.763144 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0f413-1291-4720-bcb5-ec57d7decea0 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:10.768642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 450/909] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:50:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:10.769500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0f413-1291-4720-bcb5-ec57d7decea0 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:10.775828 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b925af3-519d-46f4-96b0-9e799f9c838a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:10.789867 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:10.815317 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7065c-7653-4557-8834-65ff19c3a473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:10.816320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7065c-7653-4557-8834-65ff19c3a473 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:10.816612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7065c-7653-4557-8834-65ff19c3a473 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:10.816896 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7065c-7653-4557-8834-65ff19c3a473 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:10.817603 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7065c-7653-4557-8834-65ff19c3a473 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:10.817906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7065c-7653-4557-8834-65ff19c3a473 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:10.819320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7065c-7653-4557-8834-65ff19c3a473 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:10.824256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 460/910] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 14 18:50:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:10.825038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7065c-7653-4557-8834-65ff19c3a473 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:10.885128 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0f413-1291-4720-bcb5-ec57d7decea0 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:10.922752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd427dc-998c-4313-9082-184d06264f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:10.923669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd427dc-998c-4313-9082-184d06264f7a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:10.924716 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd427dc-998c-4313-9082-184d06264f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:10.925141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd427dc-998c-4313-9082-184d06264f7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:10.930229 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7065c-7653-4557-8834-65ff19c3a473 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:10.932606 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd427dc-998c-4313-9082-184d06264f7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:10.934142 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd427dc-998c-4313-9082-184d06264f7a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:10.949032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-dcd427dc-998c-4313-9082-184d06264f7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:10.949032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:10.949032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:10.949032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:10.949032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:10.949032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:10.949032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:10.949032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:10.949032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:10.949032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:10.949032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:10.949032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:10.949032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:50:10.949032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:50:10.949032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:50:10.950141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:50:10.950141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:10.950141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:10.950141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:10.950141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:10.950141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:10.950141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:10.950141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:10.950141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:10.950141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:10.950141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:10.950141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:10.950141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:10.950141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:10.950141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:10.950141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:10.950141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:10.951334 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:50:10.951334 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:50:10.951334 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:50:10.951334 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:50:10.951334 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:50:10.951334 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:50:10.951334 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:10.951334 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:10.956891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 451/911] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:50:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:10.956891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd427dc-998c-4313-9082-184d06264f7a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:10.977322 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:10.985730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4e57e-7aac-4593-aca5-e25c26024e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:10.988324 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4e57e-7aac-4593-aca5-e25c26024e72 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:10.988324 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4e57e-7aac-4593-aca5-e25c26024e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:10.988324 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4e57e-7aac-4593-aca5-e25c26024e72 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:10.988736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4e57e-7aac-4593-aca5-e25c26024e72 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:10.988736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4e57e-7aac-4593-aca5-e25c26024e72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:10.990113 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4e57e-7aac-4593-aca5-e25c26024e72 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:11.002864 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 461/912] 10.222.0.69 () {56 vars in 1114 bytes} [Thu Jul 14 18:50:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:11.002864 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4e57e-7aac-4593-aca5-e25c26024e72 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:11.040398 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:11.066119 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd427dc-998c-4313-9082-184d06264f7a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:11.102062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4e57e-7aac-4593-aca5-e25c26024e72 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:11.105839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21376e94-3d35-474d-b20e-b16387f59ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:11.106639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21376e94-3d35-474d-b20e-b16387f59ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:11.107033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21376e94-3d35-474d-b20e-b16387f59ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:11.107446 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21376e94-3d35-474d-b20e-b16387f59ac1 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:11.108263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21376e94-3d35-474d-b20e-b16387f59ac1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:11.108680 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21376e94-3d35-474d-b20e-b16387f59ac1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:11.110162 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21376e94-3d35-474d-b20e-b16387f59ac1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:11.127192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 452/913] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 14 18:50:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:11.127192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21376e94-3d35-474d-b20e-b16387f59ac1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:11.139229 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a1cd5-1e62-447e-aff6-cc0e75eb5a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:11.139910 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a1cd5-1e62-447e-aff6-cc0e75eb5a00 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:11.140202 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a1cd5-1e62-447e-aff6-cc0e75eb5a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:11.140487 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a1cd5-1e62-447e-aff6-cc0e75eb5a00 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:11.141271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a1cd5-1e62-447e-aff6-cc0e75eb5a00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:11.141592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a1cd5-1e62-447e-aff6-cc0e75eb5a00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:11.143159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a1cd5-1e62-447e-aff6-cc0e75eb5a00 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:11.155938 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 462/914] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:50:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:11.156717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a1cd5-1e62-447e-aff6-cc0e75eb5a00 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:11.214919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:11.232119 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21376e94-3d35-474d-b20e-b16387f59ac1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:11.268715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e841a-9ab8-4449-a8d1-b91d20265f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:11.269314 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a1cd5-1e62-447e-aff6-cc0e75eb5a00 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:11.269527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e841a-9ab8-4449-a8d1-b91d20265f19 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:11.269886 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e841a-9ab8-4449-a8d1-b91d20265f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:11.270245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e841a-9ab8-4449-a8d1-b91d20265f19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:11.271062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e841a-9ab8-4449-a8d1-b91d20265f19 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:11.272483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e841a-9ab8-4449-a8d1-b91d20265f19 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:11.280768 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:11.304951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74761e2-deff-4e64-bf32-3899847b537f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:11.305611 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74761e2-deff-4e64-bf32-3899847b537f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:11.305898 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74761e2-deff-4e64-bf32-3899847b537f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:11.306174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74761e2-deff-4e64-bf32-3899847b537f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:11.306956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74761e2-deff-4e64-bf32-3899847b537f None admin] RBAC: Adding query filter params (interface=public, service_id=6bf476953a7a4a11823f6761e2a0e96c, region_id=RegionOne) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:11.307307 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74761e2-deff-4e64-bf32-3899847b537f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:11.308673 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74761e2-deff-4e64-bf32-3899847b537f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:11.313805 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 463/915] 10.222.0.69 () {56 vars in 1251 bytes} [Thu Jul 14 18:50:11 2022] GET /identity/v3/endpoints?service_id=6bf476953a7a4a11823f6761e2a0e96c&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:11.314489 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74761e2-deff-4e64-bf32-3899847b537f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:11.393292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 453/916] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 14 18:50:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:50:11.394676 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e841a-9ab8-4449-a8d1-b91d20265f19 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:11.418009 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74761e2-deff-4e64-bf32-3899847b537f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:11.437660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:11.464086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1952f2-fada-40de-96c5-034c68cd03c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:11.465068 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1952f2-fada-40de-96c5-034c68cd03c0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:11.465555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1952f2-fada-40de-96c5-034c68cd03c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:11.466205 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1952f2-fada-40de-96c5-034c68cd03c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:11.467588 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1952f2-fada-40de-96c5-034c68cd03c0 None admin] RBAC: Adding query filter params (role.id=4def7095cc264443ab935a01966235a0, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=4ef0fad9a71d43bfbaba9c87dfed41c4) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:11.468340 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1952f2-fada-40de-96c5-034c68cd03c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:11.469997 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1952f2-fada-40de-96c5-034c68cd03c0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:11.477285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 464/917] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:50:11 2022] GET /identity/v3/role_assignments?user.id=4ef0fad9a71d43bfbaba9c87dfed41c4&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=4def7095cc264443ab935a01966235a0 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:11.540457 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e841a-9ab8-4449-a8d1-b91d20265f19 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:11.584889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebb65f-b160-4e11-b11e-31129a304703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:11.584889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebb65f-b160-4e11-b11e-31129a304703 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:11.584889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebb65f-b160-4e11-b11e-31129a304703 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:11.584889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebb65f-b160-4e11-b11e-31129a304703 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:11.585317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebb65f-b160-4e11-b11e-31129a304703 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:11.586723 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebb65f-b160-4e11-b11e-31129a304703 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:11.595651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 454/918] 10.222.0.69 () {56 vars in 1087 bytes} [Thu Jul 14 18:50:11 2022] GET /identity/v3/services => generated 1497 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:50:12.084499 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb608327-4772-419c-91d4-bec024ae5b89 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:12.085002 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb608327-4772-419c-91d4-bec024ae5b89 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:12.085107 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb608327-4772-419c-91d4-bec024ae5b89 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:12.086702 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 465/919] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:12.091914 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777d329-38bd-4b2c-9fde-7dc47c4608f3 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:12.092303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777d329-38bd-4b2c-9fde-7dc47c4608f3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:12.092491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777d329-38bd-4b2c-9fde-7dc47c4608f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:12.110183 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-1777d329-38bd-4b2c-9fde-7dc47c4608f3 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:12.111453 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1777d329-38bd-4b2c-9fde-7dc47c4608f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:12.167980 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:12.182488 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 455/920] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:12.204716 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb608327-4772-419c-91d4-bec024ae5b89 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:12.319034 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb608327-4772-419c-91d4-bec024ae5b89 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:12.354313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b584307-b08a-4778-817f-aeb1b41cfa91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:12.355123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b584307-b08a-4778-817f-aeb1b41cfa91 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:12.355429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b584307-b08a-4778-817f-aeb1b41cfa91 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:12.355701 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b584307-b08a-4778-817f-aeb1b41cfa91 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:12.356348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b584307-b08a-4778-817f-aeb1b41cfa91 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:12.357718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b584307-b08a-4778-817f-aeb1b41cfa91 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:12.369861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-4b584307-b08a-4778-817f-aeb1b41cfa91 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 18:50:12.369861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:12.369861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:50:12.369861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:50:12.369861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:50:12.369861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:50:12.369861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:50:12.369861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:12.369861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 18:50:12.369861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:12.369861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:12.369861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:12.369861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:12.369861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:12.369861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:12.369861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:12.369861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:12.370841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:12.370841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:12.370841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:12.370841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:12.370841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:12.370841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:12.370841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:50:12.370841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:50:12.370841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:50:12.370841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:50:12.370841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:12.370841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:12.370841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:12.370841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:12.370841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:12.370841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:12.371697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:12.371697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:12.371697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:12.371697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:12.371697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:12.371697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:12.371697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:12.371697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:12.371697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:12.371697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:12.371697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:50:12.371697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:12.371697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 18:50:12.371697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:12.371697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 466/921] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 14 18:50:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:12.379626 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1777d329-38bd-4b2c-9fde-7dc47c4608f3 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:12.478607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1777d329-38bd-4b2c-9fde-7dc47c4608f3 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:12.512965 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50aa33e-2184-4d8f-b4d3-2d527d97363d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:12.513634 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50aa33e-2184-4d8f-b4d3-2d527d97363d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:12.513923 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50aa33e-2184-4d8f-b4d3-2d527d97363d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:12.514202 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50aa33e-2184-4d8f-b4d3-2d527d97363d None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:12.515278 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50aa33e-2184-4d8f-b4d3-2d527d97363d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:12.515591 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50aa33e-2184-4d8f-b4d3-2d527d97363d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:12.516970 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50aa33e-2184-4d8f-b4d3-2d527d97363d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:12.528353 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b584307-b08a-4778-817f-aeb1b41cfa91 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:12.529201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 456/922] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 14 18:50:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:12.573972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:12.639337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b584307-b08a-4778-817f-aeb1b41cfa91 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:12.676027 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b878020-2261-4610-83ba-6f16314d1ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:12.676702 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b878020-2261-4610-83ba-6f16314d1ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:12.677055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b878020-2261-4610-83ba-6f16314d1ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:12.677339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b878020-2261-4610-83ba-6f16314d1ed5 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:12.678108 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b878020-2261-4610-83ba-6f16314d1ed5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:12.678425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b878020-2261-4610-83ba-6f16314d1ed5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:12.679850 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b878020-2261-4610-83ba-6f16314d1ed5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:12.685244 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 467/923] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 14 18:50:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:12.691348 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50aa33e-2184-4d8f-b4d3-2d527d97363d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:12.747584 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:12.803588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50aa33e-2184-4d8f-b4d3-2d527d97363d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:12.838516 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b868ca-90a7-4c81-8a50-1b9780196f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:12.839114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b868ca-90a7-4c81-8a50-1b9780196f35 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:12.839433 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b868ca-90a7-4c81-8a50-1b9780196f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:12.839715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b868ca-90a7-4c81-8a50-1b9780196f35 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:12.840555 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b868ca-90a7-4c81-8a50-1b9780196f35 None admin] RBAC: Adding query filter params (interface=public, service_id=a148c45177cc4695a993f85768bfb707, region_id=RegionOne) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:12.840880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b868ca-90a7-4c81-8a50-1b9780196f35 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:12.842368 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b868ca-90a7-4c81-8a50-1b9780196f35 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:12.847910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 457/924] 10.222.0.69 () {56 vars in 1251 bytes} [Thu Jul 14 18:50:12 2022] GET /identity/v3/endpoints?service_id=a148c45177cc4695a993f85768bfb707&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:12.853112 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b878020-2261-4610-83ba-6f16314d1ed5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:12.957476 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b878020-2261-4610-83ba-6f16314d1ed5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:12.959978 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:12.998178 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34854a8-4533-4e20-ab09-a678b662369f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:12.999232 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34854a8-4533-4e20-ab09-a678b662369f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:12.999745 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34854a8-4533-4e20-ab09-a678b662369f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:13.000242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34854a8-4533-4e20-ab09-a678b662369f None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:13.001129 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34854a8-4533-4e20-ab09-a678b662369f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:13.002751 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34854a8-4533-4e20-ab09-a678b662369f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:13.010009 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 468/925] 10.222.0.69 () {56 vars in 1087 bytes} [Thu Jul 14 18:50:12 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:50:13.556781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf802bb-e283-4ccb-bf49-1fb8d24dd5a9 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:13.557297 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf802bb-e283-4ccb-bf49-1fb8d24dd5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:13.557407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf802bb-e283-4ccb-bf49-1fb8d24dd5a9 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:13.558962 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 458/926] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:13.565201 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88215dbc-b204-4977-8dc5-b6c518f5124f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:13.565613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88215dbc-b204-4977-8dc5-b6c518f5124f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:13.565820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88215dbc-b204-4977-8dc5-b6c518f5124f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:13.583211 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-88215dbc-b204-4977-8dc5-b6c518f5124f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:13.584403 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-88215dbc-b204-4977-8dc5-b6c518f5124f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:13.593383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac7f76-cc06-4365-bb75-679ad89a5cb7 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:13.593678 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac7f76-cc06-4365-bb75-679ad89a5cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:13.593966 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac7f76-cc06-4365-bb75-679ad89a5cb7 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:13.595631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 459/927] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:50:13.600592 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f4a5d-4f68-4e03-98fe-0ed4e771347e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:13.600886 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f4a5d-4f68-4e03-98fe-0ed4e771347e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:13.601170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f4a5d-4f68-4e03-98fe-0ed4e771347e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:13.618068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-be8f4a5d-4f68-4e03-98fe-0ed4e771347e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:13.619300 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-be8f4a5d-4f68-4e03-98fe-0ed4e771347e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:13.663892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 469/928] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:13.665249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60416e-ff59-4ed6-b73b-65002e4f3ba2 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:13.665523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60416e-ff59-4ed6-b73b-65002e4f3ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:13.665801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60416e-ff59-4ed6-b73b-65002e4f3ba2 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:13.671077 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 470/929] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:13.672241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4fa56e-2c7a-425a-a390-7a1d4041c91f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:13.672528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4fa56e-2c7a-425a-a390-7a1d4041c91f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:13.672809 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4fa56e-2c7a-425a-a390-7a1d4041c91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:13.690899 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-8e4fa56e-2c7a-425a-a390-7a1d4041c91f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:13.691755 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 460/930] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:13.692125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8e4fa56e-2c7a-425a-a390-7a1d4041c91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:13.696079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f4a5d-4f68-4e03-98fe-0ed4e771347e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:13.749351 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:13.765083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 471/931] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:13.766107 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4fa56e-2c7a-425a-a390-7a1d4041c91f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:13.801336 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:13.822863 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f4a5d-4f68-4e03-98fe-0ed4e771347e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:13.863919 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8c1b0-9156-414a-814b-2b68101f86dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:13.864843 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8c1b0-9156-414a-814b-2b68101f86dd None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:13.865129 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8c1b0-9156-414a-814b-2b68101f86dd None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:13.865405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8c1b0-9156-414a-814b-2b68101f86dd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:13.873045 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4fa56e-2c7a-425a-a390-7a1d4041c91f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:13.876576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8c1b0-9156-414a-814b-2b68101f86dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:50:13.885191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-74a8c1b0-9156-414a-814b-2b68101f86dd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 18:50:13.885191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:13.885191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:13.885191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:13.885191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:13.885191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:13.885191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:13.885191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:13.885191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:13.885191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:13.885191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:13.885191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:13.885191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:50:13.885191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:50:13.885191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:50:13.885191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:50:13.886107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:50:13.886107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:50:13.886107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:50:13.886107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:50:13.886107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:13.886107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:13.886107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:13.886107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:13.886107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:13.886107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:13.886107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:13.886107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:13.886107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:13.886107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:13.886107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:13.886107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:13.886928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:13.886928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:13.886928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:13.886928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:13.886928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:50:13.886928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:50:13.886928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:50:13.886928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:50:13.886928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:50:13.886928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:50:13.886928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 18:50:13.886928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:13.886928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 461/932] 10.222.0.69 () {56 vars in 1120 bytes} [Thu Jul 14 18:50:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:13.887648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8c1b0-9156-414a-814b-2b68101f86dd None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:13.912967 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec4f75-d22b-4459-9def-ad11b8cad562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:13.913625 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec4f75-d22b-4459-9def-ad11b8cad562 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:13.913908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec4f75-d22b-4459-9def-ad11b8cad562 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:13.914178 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec4f75-d22b-4459-9def-ad11b8cad562 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:13.914867 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec4f75-d22b-4459-9def-ad11b8cad562 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:13.916326 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec4f75-d22b-4459-9def-ad11b8cad562 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:13.917700 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: WARNING keystone.server.flask.common [None req-a1ec4f75-d22b-4459-9def-ad11b8cad562 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 14 18:50:13.963241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.sql.core [None req-a1ec4f75-d22b-4459-9def-ad11b8cad562 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 18:50:13.963241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 18:50:13.963241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [parameters: {'id': '0ac15bb9552e4797be89ffd770ba91d9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 18:50:13.963241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80343) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 18:50:13.980640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-a1ec4f75-d22b-4459-9def-ad11b8cad562 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0ac15bb9552e4797be89ffd770ba91d9.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0ac15bb9552e4797be89ffd770ba91d9. Jul 14 18:50:13.980640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:13.980640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 18:50:13.980640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 18:50:13.980640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 18:50:13.980640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 18:50:13.980640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 18:50:13.980640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 18:50:13.980640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 18:50:13.980640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application conn.query(q) Jul 14 18:50:13.980640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 18:50:13.980640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 18:50:13.980640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 18:50:13.981478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application result.read() Jul 14 18:50:13.981478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 18:50:13.981478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 18:50:13.981478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 18:50:13.981478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 18:50:13.981478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 18:50:13.981478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 18:50:13.981478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 18:50:13.981478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 18:50:13.981478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 18:50:13.981478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:13.981478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 18:50:13.981478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:13.981478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:13.981478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 18:50:13.981478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 18:50:13.981478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 14 18:50:13.981478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 14 18:50:13.982451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 18:50:13.982451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application next(self.gen) Jul 14 18:50:13.982451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 18:50:13.982451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application yield resource Jul 14 18:50:13.982451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 18:50:13.982451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application next(self.gen) Jul 14 18:50:13.982451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 18:50:13.982451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 18:50:13.982451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 18:50:13.982451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 18:50:13.982451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 18:50:13.982451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise self.value Jul 14 18:50:13.982451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 18:50:13.982451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 18:50:13.982451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 18:50:13.982451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application session.commit() Jul 14 18:50:13.982451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 18:50:13.983972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 18:50:13.983972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 18:50:13.983972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 18:50:13.983972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 18:50:13.983972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.session.flush() Jul 14 18:50:13.983972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 18:50:13.983972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 18:50:13.983972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 18:50:13.983972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 18:50:13.983972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 18:50:13.983972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application compat.raise_( Jul 14 18:50:13.983972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 18:50:13.983972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception Jul 14 18:50:13.983972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 18:50:13.983972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 18:50:13.983972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 18:50:13.983972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 18:50:13.984870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 18:50:13.984870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 18:50:13.984870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 18:50:13.984870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 18:50:13.984870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 18:50:13.984870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 18:50:13.984870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 18:50:13.984870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 18:50:13.984870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 18:50:13.984870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 18:50:13.984870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 18:50:13.984870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 18:50:13.984870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 18:50:13.984870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 18:50:13.984870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 18:50:13.985653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 18:50:13.985653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 18:50:13.985653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception Jul 14 18:50:13.985653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 18:50:13.985653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 18:50:13.985653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 18:50:13.985653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 18:50:13.985653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 18:50:13.985653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 18:50:13.985653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 18:50:13.985653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application conn.query(q) Jul 14 18:50:13.985653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 18:50:13.985653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 18:50:13.985653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 18:50:13.985653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application result.read() Jul 14 18:50:13.985653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 18:50:13.985653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 18:50:13.987191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 18:50:13.987191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 18:50:13.987191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 18:50:13.987191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 18:50:13.987191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 18:50:13.987191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 18:50:13.987191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 18:50:13.987191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 18:50:13.987191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [parameters: {'id': '0ac15bb9552e4797be89ffd770ba91d9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 18:50:13.987191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 18:50:13.987191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:13.987191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:13.987191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:13.987191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:13.987191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 14 18:50:13.987975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 14 18:50:13.987975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 18:50:13.987975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 18:50:13.987975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 14 18:50:13.987975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:13.987975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:13.987975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:13.987975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:13.987975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:13.987975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:13.987975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:13.987975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:13.987975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:13.987975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:13.987975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:13.987975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:13.987975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:13.988943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:13.988943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 14 18:50:13.988943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 14 18:50:13.988943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:13.988943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:13.988943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 14 18:50:13.988943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 14 18:50:13.988943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0ac15bb9552e4797be89ffd770ba91d9. Jul 14 18:50:13.988943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:13.988943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 472/933] 10.222.0.69 () {60 vars in 1140 bytes} [Thu Jul 14 18:50:13 2022] POST /identity/v3/projects => generated 262 bytes in 216 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:13.988943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec4f75-d22b-4459-9def-ad11b8cad562 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:14.007781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8c1b0-9156-414a-814b-2b68101f86dd None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:14.015928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:14.045313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08bd5e8-f5da-43fb-ac78-29218f0b180c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:14.046134 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08bd5e8-f5da-43fb-ac78-29218f0b180c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:14.046419 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08bd5e8-f5da-43fb-ac78-29218f0b180c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:14.046751 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08bd5e8-f5da-43fb-ac78-29218f0b180c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:14.047448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08bd5e8-f5da-43fb-ac78-29218f0b180c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:14.048959 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08bd5e8-f5da-43fb-ac78-29218f0b180c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:14.061832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b08bd5e8-f5da-43fb-ac78-29218f0b180c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 18:50:14.061832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:14.061832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:50:14.061832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:50:14.061832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:50:14.061832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:50:14.061832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:50:14.061832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:14.061832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 18:50:14.061832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:14.061832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:14.061832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:14.061832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:14.061832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:14.061832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:14.061832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:14.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:14.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:14.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:14.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:14.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:14.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:14.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:14.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:50:14.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:50:14.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:50:14.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:50:14.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:14.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:14.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:14.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:14.062839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:14.063775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:14.063775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:14.063775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:14.063775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:14.063775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:14.063775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:14.063775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:14.063775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:14.063775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:14.063775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:14.063775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:14.063775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:50:14.063775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:14.063775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 18:50:14.063775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:14.063775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 462/934] 10.222.0.69 () {56 vars in 1132 bytes} [Thu Jul 14 18:50:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:14.065246 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08bd5e8-f5da-43fb-ac78-29218f0b180c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:14.084983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:14.102613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec4f75-d22b-4459-9def-ad11b8cad562 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:14.137820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccd417-78a5-443b-b04a-3d23ffdd44ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:14.138516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccd417-78a5-443b-b04a-3d23ffdd44ec None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:14.138800 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccd417-78a5-443b-b04a-3d23ffdd44ec None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:14.139075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccd417-78a5-443b-b04a-3d23ffdd44ec None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:14.139879 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccd417-78a5-443b-b04a-3d23ffdd44ec None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:14.140186 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccd417-78a5-443b-b04a-3d23ffdd44ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:14.141828 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccd417-78a5-443b-b04a-3d23ffdd44ec None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:14.154406 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 473/935] 10.222.0.69 () {56 vars in 1115 bytes} [Thu Jul 14 18:50:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:14.155759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccd417-78a5-443b-b04a-3d23ffdd44ec None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:14.182574 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08bd5e8-f5da-43fb-ac78-29218f0b180c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:14.219662 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23689fb-2011-4ea2-9e34-5e320d784097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:14.220597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23689fb-2011-4ea2-9e34-5e320d784097 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:14.221591 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23689fb-2011-4ea2-9e34-5e320d784097 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:14.221961 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23689fb-2011-4ea2-9e34-5e320d784097 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:14.229714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23689fb-2011-4ea2-9e34-5e320d784097 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:14.231278 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23689fb-2011-4ea2-9e34-5e320d784097 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:14.246401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-d23689fb-2011-4ea2-9e34-5e320d784097 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:14.246401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:14.246401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:14.246401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:14.246401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:14.246401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:14.246401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:14.246401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:14.246401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:14.246401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:14.246401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:14.246401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:14.246401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:50:14.246401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:50:14.246401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:50:14.247561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:50:14.247561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:14.247561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:14.247561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:14.247561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:14.247561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:14.247561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:14.247561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:14.247561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:14.247561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:14.247561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:14.247561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:14.247561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:14.247561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:14.247561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:14.247561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:14.247561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:14.248619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:50:14.248619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:50:14.248619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:50:14.248619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:50:14.248619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:50:14.248619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:50:14.248619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:14.248619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:14.254482 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 463/936] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:50:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:14.254482 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23689fb-2011-4ea2-9e34-5e320d784097 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:14.263464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccd417-78a5-443b-b04a-3d23ffdd44ec None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:14.284416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:14.298575 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980a18d-d1f2-4446-bc65-24e5dfb3b636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:14.299515 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980a18d-d1f2-4446-bc65-24e5dfb3b636 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:14.299888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980a18d-d1f2-4446-bc65-24e5dfb3b636 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:14.300238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980a18d-d1f2-4446-bc65-24e5dfb3b636 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:14.301016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980a18d-d1f2-4446-bc65-24e5dfb3b636 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:14.301404 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980a18d-d1f2-4446-bc65-24e5dfb3b636 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:14.302854 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980a18d-d1f2-4446-bc65-24e5dfb3b636 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:14.312465 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 474/937] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 14 18:50:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:14.312465 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980a18d-d1f2-4446-bc65-24e5dfb3b636 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:14.322911 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:14.365104 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23689fb-2011-4ea2-9e34-5e320d784097 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:14.402301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2760e1-5ed0-4381-adbb-2c58e3c582eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:14.403150 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2760e1-5ed0-4381-adbb-2c58e3c582eb None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:14.403464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2760e1-5ed0-4381-adbb-2c58e3c582eb None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:14.403892 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2760e1-5ed0-4381-adbb-2c58e3c582eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:14.413063 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980a18d-d1f2-4446-bc65-24e5dfb3b636 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:14.414525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2760e1-5ed0-4381-adbb-2c58e3c582eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:14.416001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2760e1-5ed0-4381-adbb-2c58e3c582eb None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:14.434164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-9a2760e1-5ed0-4381-adbb-2c58e3c582eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:14.434164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:14.434164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:14.434164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:14.434164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:14.434164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:14.434164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:14.434164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:14.434164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:14.434164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:14.434164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:14.434164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:14.434164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:14.434164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:14.434164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:14.435046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:14.435046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:14.435046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:14.435046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:14.435046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:14.435046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:14.435046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:14.435046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:14.435046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:14.435046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:14.435046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:14.435046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:14.435046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:14.435046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:14.435046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:14.435046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:14.435046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:14.435994 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:14.435994 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:14.435994 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:14.435994 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:14.435994 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 464/938] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:14.436264 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2760e1-5ed0-4381-adbb-2c58e3c582eb None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:14.451657 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386c944-dd7e-4125-bae9-5913a58fb73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:14.452318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386c944-dd7e-4125-bae9-5913a58fb73e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:14.452622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386c944-dd7e-4125-bae9-5913a58fb73e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:14.452908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386c944-dd7e-4125-bae9-5913a58fb73e None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:14.453620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386c944-dd7e-4125-bae9-5913a58fb73e None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:14.453928 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386c944-dd7e-4125-bae9-5913a58fb73e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:14.455908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386c944-dd7e-4125-bae9-5913a58fb73e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:14.473721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 475/939] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 14 18:50:14 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:14.474471 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386c944-dd7e-4125-bae9-5913a58fb73e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:14.548704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:14.570945 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:14.574474 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2760e1-5ed0-4381-adbb-2c58e3c582eb None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:14.607213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386c944-dd7e-4125-bae9-5913a58fb73e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:14.623343 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1cc94-a35b-4c91-b392-be3680cc2abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:14.624724 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1cc94-a35b-4c91-b392-be3680cc2abb None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:14.625438 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1cc94-a35b-4c91-b392-be3680cc2abb None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:14.625907 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1cc94-a35b-4c91-b392-be3680cc2abb None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:14.627522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1cc94-a35b-4c91-b392-be3680cc2abb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:14.628307 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1cc94-a35b-4c91-b392-be3680cc2abb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:14.629884 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1cc94-a35b-4c91-b392-be3680cc2abb None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:14.636785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 465/940] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 14 18:50:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:14.643331 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1cc94-a35b-4c91-b392-be3680cc2abb None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:14.649378 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720a619-4673-45bd-80ff-207b2a92541d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:14.649823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720a619-4673-45bd-80ff-207b2a92541d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:14.650118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720a619-4673-45bd-80ff-207b2a92541d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:14.650399 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720a619-4673-45bd-80ff-207b2a92541d None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:14.651127 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720a619-4673-45bd-80ff-207b2a92541d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:14.651484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720a619-4673-45bd-80ff-207b2a92541d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:14.653143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720a619-4673-45bd-80ff-207b2a92541d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:14.666740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 476/941] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:14.672849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720a619-4673-45bd-80ff-207b2a92541d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:14.786125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1cc94-a35b-4c91-b392-be3680cc2abb None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:14.796736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720a619-4673-45bd-80ff-207b2a92541d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:14.827802 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:14.834965 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f5251-5628-4a54-aec6-03f7cd33e2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:14.835132 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635fb7f-dc89-4d6f-992b-267e4978812a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:14.835671 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f5251-5628-4a54-aec6-03f7cd33e2c3 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:14.835883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635fb7f-dc89-4d6f-992b-267e4978812a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:14.835986 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f5251-5628-4a54-aec6-03f7cd33e2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:14.836167 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635fb7f-dc89-4d6f-992b-267e4978812a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:14.836265 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f5251-5628-4a54-aec6-03f7cd33e2c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:14.836441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635fb7f-dc89-4d6f-992b-267e4978812a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:14.836951 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f5251-5628-4a54-aec6-03f7cd33e2c3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:14.838742 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f5251-5628-4a54-aec6-03f7cd33e2c3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:14.844283 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635fb7f-dc89-4d6f-992b-267e4978812a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:14.845799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635fb7f-dc89-4d6f-992b-267e4978812a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:14.861268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-1635fb7f-dc89-4d6f-992b-267e4978812a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 18:50:14.861268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:14.861268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:14.861268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:14.861268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:14.861268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:14.861268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:14.861268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:14.861268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:14.861268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:14.861268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:14.861268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:14.861268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:50:14.861268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:50:14.861268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:50:14.861268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:50:14.862268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:14.862268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:14.862268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:50:14.862268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:14.862268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:50:14.862268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:14.862268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:14.862268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:14.862268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:14.862268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:14.862268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:14.862268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:14.862268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:14.862268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:14.862268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:14.862268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:14.862268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:14.862268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:14.863231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:14.863231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:14.863231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:50:14.863231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:50:14.863231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:50:14.863231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:50:14.863231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:50:14.863231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:50:14.863231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 18:50:14.863231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:14.863231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 477/942] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 14 18:50:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:14.868486 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635fb7f-dc89-4d6f-992b-267e4978812a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:14.915884 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 466/943] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 14 18:50:14 2022] POST /identity/v3/endpoints => generated 347 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:50:14.985902 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635fb7f-dc89-4d6f-992b-267e4978812a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:15.034311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d91236-1b03-42a1-b35f-9e2ef1594112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:15.035012 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d91236-1b03-42a1-b35f-9e2ef1594112 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:15.035549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d91236-1b03-42a1-b35f-9e2ef1594112 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:15.035832 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d91236-1b03-42a1-b35f-9e2ef1594112 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:15.036707 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d91236-1b03-42a1-b35f-9e2ef1594112 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:15.037018 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d91236-1b03-42a1-b35f-9e2ef1594112 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:15.038518 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d91236-1b03-42a1-b35f-9e2ef1594112 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:15.059893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 478/944] 10.222.0.69 () {56 vars in 1149 bytes} [Thu Jul 14 18:50:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:15.066230 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f5251-5628-4a54-aec6-03f7cd33e2c3 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:15.069690 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:15.069840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ef172-57a3-4ee3-a6c2-32d9f30968f9 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:15.070148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ef172-57a3-4ee3-a6c2-32d9f30968f9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:15.070443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ef172-57a3-4ee3-a6c2-32d9f30968f9 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:15.071896 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 479/945] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:50:15.077516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37929097-a903-4cae-b570-c9b08e8df5f5 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:15.077811 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37929097-a903-4cae-b570-c9b08e8df5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:15.078103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37929097-a903-4cae-b570-c9b08e8df5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:15.101409 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-37929097-a903-4cae-b570-c9b08e8df5f5 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:15.103458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-37929097-a903-4cae-b570-c9b08e8df5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:15.159130 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:15.191437 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 480/946] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:15.199958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f5251-5628-4a54-aec6-03f7cd33e2c3 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:15.213003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37929097-a903-4cae-b570-c9b08e8df5f5 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:15.239846 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966d88d-642f-4c94-aa56-2fdf8fa6663c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:15.240663 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966d88d-642f-4c94-aa56-2fdf8fa6663c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:15.240951 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966d88d-642f-4c94-aa56-2fdf8fa6663c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:15.241238 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966d88d-642f-4c94-aa56-2fdf8fa6663c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:15.255319 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b966d88d-642f-4c94-aa56-2fdf8fa6663c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:15.256788 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b966d88d-642f-4c94-aa56-2fdf8fa6663c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:15.274664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b966d88d-642f-4c94-aa56-2fdf8fa6663c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:15.274664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:15.274664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:15.274664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:15.274664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:15.274664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:15.274664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:15.274664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:15.274664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:15.274664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:15.274664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:15.274664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:15.274664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:15.274664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:15.274664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:15.275646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:15.275646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:15.275646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:15.275646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:15.275646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:15.275646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:15.275646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:15.275646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:15.275646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:15.275646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:15.275646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:15.275646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:15.275646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:15.275646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:15.275646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:15.275646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:15.275646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:15.276518 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:15.276518 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:15.276518 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:15.276518 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:15.276518 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 467/947] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:15.282848 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966d88d-642f-4c94-aa56-2fdf8fa6663c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:15.336220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37929097-a903-4cae-b570-c9b08e8df5f5 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:15.343021 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:15.374977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2bf29-75ec-4eea-a865-dc4605be8b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:15.376277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2bf29-75ec-4eea-a865-dc4605be8b4d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:15.377385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2bf29-75ec-4eea-a865-dc4605be8b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:15.377889 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2bf29-75ec-4eea-a865-dc4605be8b4d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:15.378985 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2bf29-75ec-4eea-a865-dc4605be8b4d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:15.381083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2bf29-75ec-4eea-a865-dc4605be8b4d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:15.389998 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966d88d-642f-4c94-aa56-2fdf8fa6663c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:15.396859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-32b2bf29-75ec-4eea-a865-dc4605be8b4d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 18:50:15.396859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:15.396859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:50:15.396859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:50:15.396859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:50:15.396859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:50:15.396859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:50:15.396859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:15.396859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 18:50:15.396859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:15.396859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:15.396859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:15.396859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:15.396859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:15.396859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:15.396859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:15.396859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:15.398044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:15.398044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:15.398044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:15.398044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:15.398044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:15.398044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:15.398044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:50:15.398044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:50:15.398044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:50:15.398044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:50:15.398044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:15.398044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:15.398044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:15.398044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:15.398044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:15.398044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:15.399075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:15.399075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:15.399075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:15.399075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:15.399075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:15.399075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:15.399075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:15.399075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:15.399075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:15.399075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:15.399075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:50:15.399075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:15.399075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 18:50:15.399075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:15.407269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 481/948] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 14 18:50:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:15.407871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2bf29-75ec-4eea-a865-dc4605be8b4d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:15.436064 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097b01a-1744-4629-81e9-1e82c54b22c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:15.436726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097b01a-1744-4629-81e9-1e82c54b22c7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:15.437056 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097b01a-1744-4629-81e9-1e82c54b22c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:15.437337 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097b01a-1744-4629-81e9-1e82c54b22c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:15.438058 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097b01a-1744-4629-81e9-1e82c54b22c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:15.438438 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097b01a-1744-4629-81e9-1e82c54b22c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:15.440140 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097b01a-1744-4629-81e9-1e82c54b22c7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:15.457864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 468/949] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:15.458758 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097b01a-1744-4629-81e9-1e82c54b22c7 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:15.463961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:15.551974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2bf29-75ec-4eea-a865-dc4605be8b4d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:15.575816 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097b01a-1744-4629-81e9-1e82c54b22c7 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:15.588664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62219b47-543f-4436-bad8-c069c083e561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:15.589462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62219b47-543f-4436-bad8-c069c083e561 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:15.589857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62219b47-543f-4436-bad8-c069c083e561 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:15.590226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62219b47-543f-4436-bad8-c069c083e561 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:15.591028 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62219b47-543f-4436-bad8-c069c083e561 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:15.591473 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62219b47-543f-4436-bad8-c069c083e561 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:15.592935 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62219b47-543f-4436-bad8-c069c083e561 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:15.598011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:15.603270 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 482/950] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 14 18:50:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:15.605190 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62219b47-543f-4436-bad8-c069c083e561 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:15.628692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09cdfc-0f5d-4af6-94a3-e2b6f9e2eb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:15.629895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09cdfc-0f5d-4af6-94a3-e2b6f9e2eb20 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:15.630347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09cdfc-0f5d-4af6-94a3-e2b6f9e2eb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:15.630768 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09cdfc-0f5d-4af6-94a3-e2b6f9e2eb20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:15.640027 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09cdfc-0f5d-4af6-94a3-e2b6f9e2eb20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:15.642169 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09cdfc-0f5d-4af6-94a3-e2b6f9e2eb20 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:15.661220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-9c09cdfc-0f5d-4af6-94a3-e2b6f9e2eb20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:15.661220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:15.661220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:15.661220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:15.661220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:15.661220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:15.661220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:15.661220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:15.661220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:15.661220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:15.661220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:15.661220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:15.661220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:50:15.661220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:50:15.661220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:50:15.662352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:50:15.662352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:15.662352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:15.662352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:15.662352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:15.662352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:15.662352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:15.662352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:15.662352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:15.662352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:15.662352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:15.662352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:15.662352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:15.662352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:15.662352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:15.662352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:15.662352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:15.663665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:50:15.663665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:50:15.663665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:50:15.663665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:50:15.663665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:50:15.663665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:50:15.663665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:15.663665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:15.671102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 469/951] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:50:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:15.672117 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09cdfc-0f5d-4af6-94a3-e2b6f9e2eb20 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:15.744796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62219b47-543f-4436-bad8-c069c083e561 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:15.746844 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:15.781873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90430d3e-0006-44e9-8666-d905ea6d411a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:15.782977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90430d3e-0006-44e9-8666-d905ea6d411a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:15.784097 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90430d3e-0006-44e9-8666-d905ea6d411a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:15.784598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90430d3e-0006-44e9-8666-d905ea6d411a None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:15.785499 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90430d3e-0006-44e9-8666-d905ea6d411a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:15.786037 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90430d3e-0006-44e9-8666-d905ea6d411a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:15.787698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90430d3e-0006-44e9-8666-d905ea6d411a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:15.798330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 483/952] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 14 18:50:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:15.799478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90430d3e-0006-44e9-8666-d905ea6d411a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:15.808659 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09cdfc-0f5d-4af6-94a3-e2b6f9e2eb20 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:15.856205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8009222-f95a-44c7-8cf2-264c307e5184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:15.857050 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8009222-f95a-44c7-8cf2-264c307e5184 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:15.857438 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8009222-f95a-44c7-8cf2-264c307e5184 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:15.857727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8009222-f95a-44c7-8cf2-264c307e5184 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:15.858519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8009222-f95a-44c7-8cf2-264c307e5184 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:15.858838 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8009222-f95a-44c7-8cf2-264c307e5184 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:15.861272 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8009222-f95a-44c7-8cf2-264c307e5184 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:15.882598 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 470/953] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:50:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:15.888127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8009222-f95a-44c7-8cf2-264c307e5184 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:15.914056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90430d3e-0006-44e9-8666-d905ea6d411a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:15.925762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:15.959931 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48004e0d-5b86-4a53-befc-cd0c436ce805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:15.960929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48004e0d-5b86-4a53-befc-cd0c436ce805 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:15.961437 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48004e0d-5b86-4a53-befc-cd0c436ce805 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:15.961911 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48004e0d-5b86-4a53-befc-cd0c436ce805 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:15.962844 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48004e0d-5b86-4a53-befc-cd0c436ce805 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:15.964532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48004e0d-5b86-4a53-befc-cd0c436ce805 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:15.999629 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8009222-f95a-44c7-8cf2-264c307e5184 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:16.036759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242206fd-21b4-4cb5-a7de-79afe478b22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:16.037545 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242206fd-21b4-4cb5-a7de-79afe478b22f None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:16.037838 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242206fd-21b4-4cb5-a7de-79afe478b22f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:16.038122 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242206fd-21b4-4cb5-a7de-79afe478b22f None admin] PATH_INFO: `/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/4ef0fad9a71d43bfbaba9c87dfed41c4/roles/4def7095cc264443ab935a01966235a0` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:16.040583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242206fd-21b4-4cb5-a7de-79afe478b22f None admin] RBAC: Authorizing `identity:create_grant(project_id=6003dba8ddd247e7842d28ea7ca5238c, user_id=4ef0fad9a71d43bfbaba9c87dfed41c4, role_id=4def7095cc264443ab935a01966235a0)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:16.042030 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242206fd-21b4-4cb5-a7de-79afe478b22f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:16.047266 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 484/954] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 14 18:50:15 2022] POST /identity/v3/endpoints => generated 349 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:50:16.123200 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 471/955] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:50:15 2022] PUT /identity/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/4ef0fad9a71d43bfbaba9c87dfed41c4/roles/4def7095cc264443ab935a01966235a0 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 18:50:16.614340 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2eb69-3cd2-42df-bece-33910f078a97 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:16.614867 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2eb69-3cd2-42df-bece-33910f078a97 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:16.615224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2eb69-3cd2-42df-bece-33910f078a97 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:16.616969 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 485/956] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:16.622607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4a97e-0e2f-4c49-aa0c-2934539e0378 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:16.623407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4a97e-0e2f-4c49-aa0c-2934539e0378 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:16.623860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4a97e-0e2f-4c49-aa0c-2934539e0378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:16.641240 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4bf4a97e-0e2f-4c49-aa0c-2934539e0378 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:16.643248 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4bf4a97e-0e2f-4c49-aa0c-2934539e0378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:16.727255 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 472/957] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:16.745083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2eb69-3cd2-42df-bece-33910f078a97 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:16.770633 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:16.852401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2eb69-3cd2-42df-bece-33910f078a97 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:16.888058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95ddb2-ef92-473c-bd91-9f1f0eebbb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:16.888973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95ddb2-ef92-473c-bd91-9f1f0eebbb6e None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:16.889402 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95ddb2-ef92-473c-bd91-9f1f0eebbb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:16.889818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95ddb2-ef92-473c-bd91-9f1f0eebbb6e None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:16.890659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95ddb2-ef92-473c-bd91-9f1f0eebbb6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:16.892282 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95ddb2-ef92-473c-bd91-9f1f0eebbb6e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:16.941294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.sql.core [None req-ca95ddb2-ef92-473c-bd91-9f1f0eebbb6e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 18:50:16.941294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 18:50:16.941294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [parameters: {'user_id': 'b90805a865cd455eb22d3652798c349a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 18:50:16.941294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80343) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 18:50:16.949332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-ca95ddb2-ef92-473c-bd91-9f1f0eebbb6e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 18:50:16.949332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:16.949332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 18:50:16.949332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 18:50:16.949332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 18:50:16.949332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 18:50:16.949332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 18:50:16.949332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 18:50:16.949332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 18:50:16.949332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application conn.query(q) Jul 14 18:50:16.949332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 18:50:16.949332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 18:50:16.949332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 18:50:16.949332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application result.read() Jul 14 18:50:16.949332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 18:50:16.950475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 18:50:16.950475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 18:50:16.950475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 18:50:16.950475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 18:50:16.950475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 18:50:16.950475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 18:50:16.950475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 18:50:16.950475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 18:50:16.950475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:16.950475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 18:50:16.950475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:16.950475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:16.950475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 18:50:16.950475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 18:50:16.950475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 14 18:50:16.950475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 14 18:50:16.950475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 18:50:16.950475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application next(self.gen) Jul 14 18:50:16.951671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 18:50:16.951671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application yield resource Jul 14 18:50:16.951671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 18:50:16.951671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application next(self.gen) Jul 14 18:50:16.951671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 18:50:16.951671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 18:50:16.951671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 18:50:16.951671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 18:50:16.951671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 18:50:16.951671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise self.value Jul 14 18:50:16.951671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 18:50:16.951671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 18:50:16.951671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 18:50:16.951671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application session.commit() Jul 14 18:50:16.951671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 18:50:16.951671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 18:50:16.951671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 18:50:16.952770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 18:50:16.952770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 18:50:16.952770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.session.flush() Jul 14 18:50:16.952770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 18:50:16.952770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 18:50:16.952770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 18:50:16.952770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 18:50:16.952770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 18:50:16.952770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application compat.raise_( Jul 14 18:50:16.952770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 18:50:16.952770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception Jul 14 18:50:16.952770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 18:50:16.952770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 18:50:16.952770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 18:50:16.952770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 18:50:16.952770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 18:50:16.952770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 18:50:16.953847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 18:50:16.953847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 18:50:16.953847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 14 18:50:16.953847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 14 18:50:16.953847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 18:50:16.953847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 18:50:16.953847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 18:50:16.953847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 18:50:16.953847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 18:50:16.953847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 18:50:16.953847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 18:50:16.953847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 18:50:16.953847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 18:50:16.953847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 18:50:16.953847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 18:50:16.953847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception Jul 14 18:50:16.954857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 18:50:16.954857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 18:50:16.954857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 18:50:16.954857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 18:50:16.954857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 18:50:16.954857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 18:50:16.954857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 18:50:16.954857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application conn.query(q) Jul 14 18:50:16.954857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 18:50:16.954857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 18:50:16.954857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 18:50:16.954857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application result.read() Jul 14 18:50:16.954857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 18:50:16.954857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 18:50:16.954857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 18:50:16.954857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 18:50:16.956467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 18:50:16.956467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 18:50:16.956467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 18:50:16.956467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 18:50:16.956467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 18:50:16.956467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 18:50:16.956467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b90805a865cd455eb22d3652798c349a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 18:50:16.956467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 18:50:16.956467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:16.956467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:16.956467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:16.956467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:16.956467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:16.956467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:16.956467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:16.957401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:16.957401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:16.957401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:16.957401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:16.957401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:16.957401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:16.957401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:16.957401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 14 18:50:16.957401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 14 18:50:16.957401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:16.957401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:16.957401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:50:16.957401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:16.957401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:50:16.957401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:16.957401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 14 18:50:16.957401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 14 18:50:16.958313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 14 18:50:16.958313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 14 18:50:16.958313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 18:50:16.958313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 18:50:16.958313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 18:50:16.958313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:16.960434 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 486/958] 10.222.0.69 () {60 vars in 1131 bytes} [Thu Jul 14 18:50:16 2022] POST /identity/v3/users => generated 160 bytes in 216 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:16.966387 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4a97e-0e2f-4c49-aa0c-2934539e0378 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:16.991239 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:17.074770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4a97e-0e2f-4c49-aa0c-2934539e0378 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:17.081006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b18e0c-b382-40c7-9aba-cabfe460fa2a None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:17.081304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b18e0c-b382-40c7-9aba-cabfe460fa2a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:17.081591 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b18e0c-b382-40c7-9aba-cabfe460fa2a None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:17.089871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 487/959] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:17 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:50:17.091134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee61d5-ec1c-4836-b6ff-2ac149783b3c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:17.091647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee61d5-ec1c-4836-b6ff-2ac149783b3c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:17.091739 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee61d5-ec1c-4836-b6ff-2ac149783b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:17.108882 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-50ee61d5-ec1c-4836-b6ff-2ac149783b3c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:17.110399 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-50ee61d5-ec1c-4836-b6ff-2ac149783b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:17.117714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41a35f-3e2b-4faa-b650-32e11b6aa89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:17.120173 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41a35f-3e2b-4faa-b650-32e11b6aa89d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:17.120632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41a35f-3e2b-4faa-b650-32e11b6aa89d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:17.121098 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41a35f-3e2b-4faa-b650-32e11b6aa89d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:17.132118 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41a35f-3e2b-4faa-b650-32e11b6aa89d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:17.134916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41a35f-3e2b-4faa-b650-32e11b6aa89d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:17.162776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-1a41a35f-3e2b-4faa-b650-32e11b6aa89d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 18:50:17.162776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:17.162776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:17.162776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:17.162776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:17.162776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:17.162776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:17.162776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:17.162776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:17.162776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:17.162776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:17.162776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:17.162776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:50:17.162776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:50:17.162776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:50:17.162776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:50:17.164115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:17.164115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:17.164115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:50:17.164115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:17.164115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:50:17.164115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:17.164115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:17.164115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:17.164115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:17.164115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:17.164115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:17.164115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:17.164115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:17.164115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:17.164115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:17.164115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:17.164115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:17.164115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:17.165341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:17.165341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:17.165341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:50:17.165341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:50:17.165341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:50:17.165341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:50:17.165341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:50:17.165341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:50:17.165341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 18:50:17.165341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:17.168313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 473/960] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:50:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:17.176139 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41a35f-3e2b-4faa-b650-32e11b6aa89d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:17.195047 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 488/961] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:17.217286 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee61d5-ec1c-4836-b6ff-2ac149783b3c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:17.233626 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:17.275025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:17.308713 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41a35f-3e2b-4faa-b650-32e11b6aa89d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:17.354380 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee61d5-ec1c-4836-b6ff-2ac149783b3c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:17.355762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77373a69-6f99-4b58-ada8-a40fb164b6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:17.356695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77373a69-6f99-4b58-ada8-a40fb164b6bd None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:17.357204 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77373a69-6f99-4b58-ada8-a40fb164b6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:17.357695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77373a69-6f99-4b58-ada8-a40fb164b6bd None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:17.358725 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77373a69-6f99-4b58-ada8-a40fb164b6bd None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:17.359300 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77373a69-6f99-4b58-ada8-a40fb164b6bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:17.360967 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77373a69-6f99-4b58-ada8-a40fb164b6bd None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:17.393268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109a4ae-596f-479a-912c-1920c80f53f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:17.394094 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109a4ae-596f-479a-912c-1920c80f53f9 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:17.394420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109a4ae-596f-479a-912c-1920c80f53f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:17.394703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109a4ae-596f-479a-912c-1920c80f53f9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:17.395415 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109a4ae-596f-479a-912c-1920c80f53f9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:17.397546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109a4ae-596f-479a-912c-1920c80f53f9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:17.400700 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 474/962] 10.222.0.69 () {56 vars in 1101 bytes} [Thu Jul 14 18:50:17 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:17.410840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-8109a4ae-596f-479a-912c-1920c80f53f9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 18:50:17.410840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:17.410840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:50:17.410840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:50:17.410840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:50:17.410840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:50:17.410840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:50:17.410840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:17.410840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 18:50:17.410840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:17.410840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:17.410840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:17.410840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:17.410840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:17.410840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:17.410840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:17.410840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:17.412030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:17.412030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:17.412030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:17.412030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:17.412030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:17.412030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:17.412030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:50:17.412030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:50:17.412030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:50:17.412030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:50:17.412030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:17.412030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:17.412030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:17.412030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:17.412030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:17.412030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:17.412964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:17.412964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:17.412964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:17.412964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:17.412964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:17.412964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:17.412964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:17.412964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:17.412964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:17.412964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:17.412964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:50:17.412964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:17.412964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 18:50:17.412964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:17.412964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 489/963] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:50:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:17.420675 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77373a69-6f99-4b58-ada8-a40fb164b6bd None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:17.540605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77373a69-6f99-4b58-ada8-a40fb164b6bd None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:17.578139 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bda3f9-e56e-45ff-a7ba-ed4d891135d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:17.578809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bda3f9-e56e-45ff-a7ba-ed4d891135d9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:17.579094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bda3f9-e56e-45ff-a7ba-ed4d891135d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:17.579420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bda3f9-e56e-45ff-a7ba-ed4d891135d9 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:17.580173 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bda3f9-e56e-45ff-a7ba-ed4d891135d9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:17.580535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bda3f9-e56e-45ff-a7ba-ed4d891135d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:17.581917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bda3f9-e56e-45ff-a7ba-ed4d891135d9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:17.587215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 475/964] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 14 18:50:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:17.592960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109a4ae-596f-479a-912c-1920c80f53f9 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:17.690667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:17.712024 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109a4ae-596f-479a-912c-1920c80f53f9 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:17.748374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5deda14-ae2f-411c-95cc-3e432c854685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:17.749048 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5deda14-ae2f-411c-95cc-3e432c854685 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:17.749333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5deda14-ae2f-411c-95cc-3e432c854685 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:17.749612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5deda14-ae2f-411c-95cc-3e432c854685 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:17.750319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5deda14-ae2f-411c-95cc-3e432c854685 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:17.750629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5deda14-ae2f-411c-95cc-3e432c854685 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:17.752176 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5deda14-ae2f-411c-95cc-3e432c854685 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:17.763197 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 490/965] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 14 18:50:17 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:18.198408 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100386f6-84ed-4d2d-9b68-4f77ea38335c None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:18.198919 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100386f6-84ed-4d2d-9b68-4f77ea38335c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:18.199020 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100386f6-84ed-4d2d-9b68-4f77ea38335c None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:18.200377 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 476/966] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:18.205900 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9208ac3-0fac-4b7d-88b3-1d653eb26ded None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:18.206204 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9208ac3-0fac-4b7d-88b3-1d653eb26ded None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:18.206492 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9208ac3-0fac-4b7d-88b3-1d653eb26ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:18.223205 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e9208ac3-0fac-4b7d-88b3-1d653eb26ded None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:18.224396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e9208ac3-0fac-4b7d-88b3-1d653eb26ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:18.251499 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42097739-57d1-44a1-bbec-7b1655bdae80 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:18.251784 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42097739-57d1-44a1-bbec-7b1655bdae80 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:18.252635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42097739-57d1-44a1-bbec-7b1655bdae80 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:18.254110 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 477/967] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:50:18.258842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626e9c8-3e80-46d0-90b2-aec407f21e87 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:18.259130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626e9c8-3e80-46d0-90b2-aec407f21e87 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:18.259386 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626e9c8-3e80-46d0-90b2-aec407f21e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:18.264286 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:18.275728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3626e9c8-3e80-46d0-90b2-aec407f21e87 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:18.279626 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3626e9c8-3e80-46d0-90b2-aec407f21e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:18.296104 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 491/968] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:18.319601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9208ac3-0fac-4b7d-88b3-1d653eb26ded None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:18.348187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 478/969] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:18.371745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626e9c8-3e80-46d0-90b2-aec407f21e87 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:18.390797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:18.420107 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9208ac3-0fac-4b7d-88b3-1d653eb26ded None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:18.459178 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ad21d-8cc3-4dd8-8c28-2404ba8cc038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:18.460013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ad21d-8cc3-4dd8-8c28-2404ba8cc038 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:18.460308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ad21d-8cc3-4dd8-8c28-2404ba8cc038 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:18.460620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ad21d-8cc3-4dd8-8c28-2404ba8cc038 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:18.461248 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ad21d-8cc3-4dd8-8c28-2404ba8cc038 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:18.462635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ad21d-8cc3-4dd8-8c28-2404ba8cc038 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:18.475376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-498ad21d-8cc3-4dd8-8c28-2404ba8cc038 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 18:50:18.475376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:18.475376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:50:18.475376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:50:18.475376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:50:18.475376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:50:18.475376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:50:18.475376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:18.475376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 18:50:18.475376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:18.475376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:18.475376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:18.475376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:18.475376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:18.475376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:18.475376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:18.475376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:18.476474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:18.476474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:18.476474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:18.476474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:18.476474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:18.476474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:18.476474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:50:18.476474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:50:18.476474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:50:18.476474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:50:18.476474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:18.476474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:18.476474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:18.476474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:18.476474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:18.476474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:18.477742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:18.477742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:18.477742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:18.477742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:18.477742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:18.477742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:18.477742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:18.477742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:18.477742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:18.477742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:18.477742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:50:18.477742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:18.477742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 18:50:18.477742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:18.477742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 492/970] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 14 18:50:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:18.482354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626e9c8-3e80-46d0-90b2-aec407f21e87 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:18.482516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ad21d-8cc3-4dd8-8c28-2404ba8cc038 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:18.515768 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:18.521017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda3663-44ec-4cf7-9f50-11cad4ed64fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:18.521839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda3663-44ec-4cf7-9f50-11cad4ed64fa None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:18.522133 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda3663-44ec-4cf7-9f50-11cad4ed64fa None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:18.522414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda3663-44ec-4cf7-9f50-11cad4ed64fa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:18.534076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda3663-44ec-4cf7-9f50-11cad4ed64fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:50:18.542416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-9bda3663-44ec-4cf7-9f50-11cad4ed64fa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 18:50:18.542416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:18.542416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:18.542416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:18.542416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:18.542416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:18.542416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:18.542416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:18.542416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:18.542416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:18.542416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:18.542416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:18.542416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:50:18.542416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:50:18.542416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:50:18.542416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:50:18.543586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:50:18.543586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:50:18.543586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:50:18.543586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:50:18.543586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:18.543586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:18.543586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:18.543586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:18.543586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:18.543586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:18.543586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:18.543586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:18.543586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:18.543586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:18.543586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:18.543586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:18.544595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:18.544595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:18.544595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:18.544595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:18.544595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:50:18.544595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:50:18.544595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:50:18.544595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:50:18.544595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:50:18.544595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:50:18.544595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 18:50:18.544595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:18.550185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 479/971] 10.222.0.69 () {56 vars in 1120 bytes} [Thu Jul 14 18:50:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:18.551280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda3663-44ec-4cf7-9f50-11cad4ed64fa None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:18.592132 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ad21d-8cc3-4dd8-8c28-2404ba8cc038 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:18.629875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d9c61-5aff-4691-ae87-64f12ce3caba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:18.631066 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d9c61-5aff-4691-ae87-64f12ce3caba None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:18.631386 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d9c61-5aff-4691-ae87-64f12ce3caba None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:18.631668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d9c61-5aff-4691-ae87-64f12ce3caba None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:18.632838 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d9c61-5aff-4691-ae87-64f12ce3caba None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:18.633159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d9c61-5aff-4691-ae87-64f12ce3caba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:18.635255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d9c61-5aff-4691-ae87-64f12ce3caba None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:18.637102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:18.645751 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 493/972] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 14 18:50:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:18.647005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d9c61-5aff-4691-ae87-64f12ce3caba None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:18.660302 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda3663-44ec-4cf7-9f50-11cad4ed64fa None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:18.697563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23d6eb-3608-4a7e-a3e2-0e8326ccee56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:18.698787 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23d6eb-3608-4a7e-a3e2-0e8326ccee56 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:18.699338 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23d6eb-3608-4a7e-a3e2-0e8326ccee56 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:18.699807 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23d6eb-3608-4a7e-a3e2-0e8326ccee56 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:18.701158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23d6eb-3608-4a7e-a3e2-0e8326ccee56 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:18.701669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23d6eb-3608-4a7e-a3e2-0e8326ccee56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:18.703674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23d6eb-3608-4a7e-a3e2-0e8326ccee56 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:18.718096 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 480/973] 10.222.0.69 () {56 vars in 1115 bytes} [Thu Jul 14 18:50:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:18.721940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23d6eb-3608-4a7e-a3e2-0e8326ccee56 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:18.753078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d9c61-5aff-4691-ae87-64f12ce3caba None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:18.778668 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:18.789084 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5f54c-769f-447e-8869-ac59f16b22f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:18.789846 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5f54c-769f-447e-8869-ac59f16b22f0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:18.790119 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5f54c-769f-447e-8869-ac59f16b22f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:18.790626 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5f54c-769f-447e-8869-ac59f16b22f0 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:18.791048 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad5f54c-769f-447e-8869-ac59f16b22f0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:18.791383 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad5f54c-769f-447e-8869-ac59f16b22f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:18.792830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad5f54c-769f-447e-8869-ac59f16b22f0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:18.803356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 494/974] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 14 18:50:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:18.804530 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5f54c-769f-447e-8869-ac59f16b22f0 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:18.826442 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23d6eb-3608-4a7e-a3e2-0e8326ccee56 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:18.864129 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8107ce-c7b6-4677-b1d0-6c18d730e610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:18.865045 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8107ce-c7b6-4677-b1d0-6c18d730e610 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:18.865361 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8107ce-c7b6-4677-b1d0-6c18d730e610 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:18.865729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8107ce-c7b6-4677-b1d0-6c18d730e610 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:18.871505 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:18.878357 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8107ce-c7b6-4677-b1d0-6c18d730e610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:18.879952 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8107ce-c7b6-4677-b1d0-6c18d730e610 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:18.897893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-3e8107ce-c7b6-4677-b1d0-6c18d730e610 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:18.897893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:18.897893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:18.897893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:18.897893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:18.897893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:18.897893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:18.897893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:18.897893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:18.897893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:18.897893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:18.897893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:18.897893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:18.897893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:18.897893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:18.898973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:18.898973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:18.898973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:18.898973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:18.898973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:18.898973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:18.898973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:18.898973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:18.898973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:18.898973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:18.898973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:18.898973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:18.898973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:18.898973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:18.898973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:18.898973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:18.898973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:18.900130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:18.900130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:18.900130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:18.900130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:18.900130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 481/975] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:18.905775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8107ce-c7b6-4677-b1d0-6c18d730e610 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:18.914289 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5f54c-769f-447e-8869-ac59f16b22f0 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:18.949662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8e36b-2ffe-4a22-aeed-627fdf98fdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:18.950319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8e36b-2ffe-4a22-aeed-627fdf98fdec None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:18.950607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8e36b-2ffe-4a22-aeed-627fdf98fdec None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:18.950883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8e36b-2ffe-4a22-aeed-627fdf98fdec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:18.951701 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8e36b-2ffe-4a22-aeed-627fdf98fdec None admin] RBAC: Adding query filter params (interface=admin, service_id=a148c45177cc4695a993f85768bfb707, region_id=RegionOne) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:18.952018 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8e36b-2ffe-4a22-aeed-627fdf98fdec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:18.953387 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8e36b-2ffe-4a22-aeed-627fdf98fdec None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:18.963977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 495/976] 10.222.0.69 () {56 vars in 1249 bytes} [Thu Jul 14 18:50:18 2022] GET /identity/v3/endpoints?service_id=a148c45177cc4695a993f85768bfb707&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:18.964964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8e36b-2ffe-4a22-aeed-627fdf98fdec None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:19.014753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8107ce-c7b6-4677-b1d0-6c18d730e610 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:19.023477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:19.057636 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062a084-addf-464d-a6b1-7e0159d57785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:19.058221 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062a084-addf-464d-a6b1-7e0159d57785 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:19.058504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062a084-addf-464d-a6b1-7e0159d57785 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:19.058833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062a084-addf-464d-a6b1-7e0159d57785 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:19.059725 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062a084-addf-464d-a6b1-7e0159d57785 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:19.060036 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062a084-addf-464d-a6b1-7e0159d57785 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:19.061398 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062a084-addf-464d-a6b1-7e0159d57785 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:19.073159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 482/977] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:19.078756 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062a084-addf-464d-a6b1-7e0159d57785 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:19.092300 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8e36b-2ffe-4a22-aeed-627fdf98fdec None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:19.125049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:19.134309 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851d1ff-ec37-42d6-b6de-d05f7b89f639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:19.134977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851d1ff-ec37-42d6-b6de-d05f7b89f639 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:19.135515 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851d1ff-ec37-42d6-b6de-d05f7b89f639 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:19.135625 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851d1ff-ec37-42d6-b6de-d05f7b89f639 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:19.136224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851d1ff-ec37-42d6-b6de-d05f7b89f639 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:19.137603 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851d1ff-ec37-42d6-b6de-d05f7b89f639 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:19.144300 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 496/978] 10.222.0.69 () {56 vars in 1087 bytes} [Thu Jul 14 18:50:18 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 18:50:19.189160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062a084-addf-464d-a6b1-7e0159d57785 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:19.228400 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361944cf-d458-440e-86fa-c700f506662d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:19.229241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361944cf-d458-440e-86fa-c700f506662d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:19.229531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361944cf-d458-440e-86fa-c700f506662d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:19.229815 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361944cf-d458-440e-86fa-c700f506662d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:19.238754 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361944cf-d458-440e-86fa-c700f506662d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:19.240273 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361944cf-d458-440e-86fa-c700f506662d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:19.256823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-361944cf-d458-440e-86fa-c700f506662d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 18:50:19.256823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:19.256823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:19.256823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:19.256823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:19.256823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:19.256823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:19.256823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:19.256823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:19.256823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:19.256823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:19.256823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:19.256823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:50:19.256823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:50:19.256823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:50:19.256823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:50:19.257873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:19.257873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:19.257873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:50:19.257873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:19.257873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:50:19.257873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:19.257873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:19.257873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:19.257873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:19.257873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:19.257873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:19.257873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:19.257873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:19.257873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:19.257873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:19.257873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:19.257873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:19.257873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:19.258805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:19.258805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:19.258805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:50:19.258805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:50:19.258805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:50:19.258805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:50:19.258805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:50:19.258805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:50:19.258805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 18:50:19.258805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:19.263236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 483/979] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 14 18:50:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:19.264270 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851d1ff-ec37-42d6-b6de-d05f7b89f639 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:19.371229 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851d1ff-ec37-42d6-b6de-d05f7b89f639 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:19.406158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad8b4e-1198-4dc8-be5c-e119963dde27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:19.406841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad8b4e-1198-4dc8-be5c-e119963dde27 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:19.407128 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad8b4e-1198-4dc8-be5c-e119963dde27 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:19.407432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad8b4e-1198-4dc8-be5c-e119963dde27 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:19.408300 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad8b4e-1198-4dc8-be5c-e119963dde27 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:19.408610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad8b4e-1198-4dc8-be5c-e119963dde27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:19.410046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad8b4e-1198-4dc8-be5c-e119963dde27 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:19.428282 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 497/980] 10.222.0.69 () {56 vars in 1149 bytes} [Thu Jul 14 18:50:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:19.433882 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361944cf-d458-440e-86fa-c700f506662d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:19.464234 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:19.538800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361944cf-d458-440e-86fa-c700f506662d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:19.573393 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3bcd0-2fa3-40e7-96d2-1050b5c3795b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:19.574049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3bcd0-2fa3-40e7-96d2-1050b5c3795b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:19.574346 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3bcd0-2fa3-40e7-96d2-1050b5c3795b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:19.574629 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3bcd0-2fa3-40e7-96d2-1050b5c3795b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:19.646046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d3bcd0-2fa3-40e7-96d2-1050b5c3795b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:19.647466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d3bcd0-2fa3-40e7-96d2-1050b5c3795b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:19.673709 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:19.755159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 484/981] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:50:19 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 18:50:19.756741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad8b4e-1198-4dc8-be5c-e119963dde27 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:19.768659 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3be0df-3f82-4143-965a-dfa9b03684d6 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:19.768971 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3be0df-3f82-4143-965a-dfa9b03684d6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:19.769231 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3be0df-3f82-4143-965a-dfa9b03684d6 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:19.770737 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 485/982] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:50:19.775994 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78bc56-749e-4bdc-89a1-82cbe84f2973 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:19.776293 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78bc56-749e-4bdc-89a1-82cbe84f2973 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:19.776565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78bc56-749e-4bdc-89a1-82cbe84f2973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:19.797485 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ca78bc56-749e-4bdc-89a1-82cbe84f2973 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:19.798703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ca78bc56-749e-4bdc-89a1-82cbe84f2973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:19.810230 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:19.864268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad8b4e-1198-4dc8-be5c-e119963dde27 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:19.869468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 486/983] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:19.891803 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78bc56-749e-4bdc-89a1-82cbe84f2973 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:19.900636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62429c81-2a5c-4ab5-a4f4-21d64e68b90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:19.901641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62429c81-2a5c-4ab5-a4f4-21d64e68b90c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:19.902189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62429c81-2a5c-4ab5-a4f4-21d64e68b90c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:19.902463 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62429c81-2a5c-4ab5-a4f4-21d64e68b90c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:19.912984 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62429c81-2a5c-4ab5-a4f4-21d64e68b90c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:19.914439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62429c81-2a5c-4ab5-a4f4-21d64e68b90c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:19.932054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-62429c81-2a5c-4ab5-a4f4-21d64e68b90c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:19.932054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:19.932054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:19.932054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:19.932054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:19.932054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:19.932054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:19.932054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:19.932054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:19.932054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:19.932054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:19.932054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:19.932054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:19.932054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:19.932054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:19.932947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:19.932947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:19.932947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:19.932947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:19.932947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:19.932947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:19.932947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:19.932947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:19.932947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:19.932947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:19.932947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:19.932947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:19.932947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:19.932947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:19.932947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:19.932947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:19.932947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:19.933890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:19.933890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:19.933890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:19.933890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:19.933890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 498/984] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:19.938635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62429c81-2a5c-4ab5-a4f4-21d64e68b90c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:19.950872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:19.997760 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78bc56-749e-4bdc-89a1-82cbe84f2973 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:20.032821 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ab4ea-d659-47d6-99d0-8335d71bd539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:20.033518 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ab4ea-d659-47d6-99d0-8335d71bd539 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:20.033811 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ab4ea-d659-47d6-99d0-8335d71bd539 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:20.034097 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ab4ea-d659-47d6-99d0-8335d71bd539 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:20.034806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ab4ea-d659-47d6-99d0-8335d71bd539 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:20.036217 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ab4ea-d659-47d6-99d0-8335d71bd539 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:20.047974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62429c81-2a5c-4ab5-a4f4-21d64e68b90c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:20.049585 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:20.082167 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278d278-8faa-4e90-b78c-517b79dc6bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:20.082837 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278d278-8faa-4e90-b78c-517b79dc6bba None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:20.083128 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278d278-8faa-4e90-b78c-517b79dc6bba None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:20.083461 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278d278-8faa-4e90-b78c-517b79dc6bba None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:20.084188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7278d278-8faa-4e90-b78c-517b79dc6bba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:20.084499 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7278d278-8faa-4e90-b78c-517b79dc6bba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:20.085995 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7278d278-8faa-4e90-b78c-517b79dc6bba None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:20.097124 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 499/985] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:20.102555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278d278-8faa-4e90-b78c-517b79dc6bba None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:20.130591 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 487/986] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 14 18:50:19 2022] POST /identity/v3/services => generated 242 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:50:20.215808 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278d278-8faa-4e90-b78c-517b79dc6bba None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:20.261694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365f842-f8d1-45a6-97e4-215350303b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:20.262533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365f842-f8d1-45a6-97e4-215350303b65 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:20.262821 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365f842-f8d1-45a6-97e4-215350303b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:20.263102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365f842-f8d1-45a6-97e4-215350303b65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:20.271115 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365f842-f8d1-45a6-97e4-215350303b65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:20.272634 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365f842-f8d1-45a6-97e4-215350303b65 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:20.288819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-5365f842-f8d1-45a6-97e4-215350303b65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:20.288819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:20.288819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:20.288819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:20.288819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:20.288819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:20.288819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:20.288819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:20.288819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:20.288819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:20.288819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:20.288819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:20.288819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:50:20.288819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:50:20.288819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:50:20.289759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:50:20.289759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:20.289759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:20.289759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:20.289759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:20.289759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:20.289759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:20.289759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:20.289759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:20.289759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:20.289759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:20.289759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:20.289759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:20.289759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:20.289759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:20.289759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:20.289759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:20.290658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:50:20.290658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:50:20.290658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:50:20.290658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:50:20.290658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:50:20.290658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:50:20.290658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:20.290658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:20.290658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 500/987] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:50:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:20.295560 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ab4ea-d659-47d6-99d0-8335d71bd539 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:20.401440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ab4ea-d659-47d6-99d0-8335d71bd539 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:20.417714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:20.436122 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e2ed9-a7b4-4c55-8fdf-c51a9a7bbaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:20.436791 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e2ed9-a7b4-4c55-8fdf-c51a9a7bbaec None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:20.437080 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e2ed9-a7b4-4c55-8fdf-c51a9a7bbaec None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:20.437359 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e2ed9-a7b4-4c55-8fdf-c51a9a7bbaec None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:20.438187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e2ed9-a7b4-4c55-8fdf-c51a9a7bbaec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:20.438615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e2ed9-a7b4-4c55-8fdf-c51a9a7bbaec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:20.440781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e2ed9-a7b4-4c55-8fdf-c51a9a7bbaec None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:20.453792 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 488/988] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:50:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:20.459700 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365f842-f8d1-45a6-97e4-215350303b65 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:20.478961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:20.561847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365f842-f8d1-45a6-97e4-215350303b65 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:20.595015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb917cd1-960e-4dcc-b143-0af4615f582b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:20.596341 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb917cd1-960e-4dcc-b143-0af4615f582b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:20.596628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb917cd1-960e-4dcc-b143-0af4615f582b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:20.596903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb917cd1-960e-4dcc-b143-0af4615f582b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:20.597730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb917cd1-960e-4dcc-b143-0af4615f582b None admin] RBAC: Adding query filter params (role.id=4def7095cc264443ab935a01966235a0, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=4ef0fad9a71d43bfbaba9c87dfed41c4) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:20.598040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb917cd1-960e-4dcc-b143-0af4615f582b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:20.599503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb917cd1-960e-4dcc-b143-0af4615f582b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:20.605644 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 501/989] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:50:20 2022] GET /identity/v3/role_assignments?user.id=4ef0fad9a71d43bfbaba9c87dfed41c4&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=4def7095cc264443ab935a01966235a0 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:21.171465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9595a9-9d6b-4f26-bbca-ded4a72c526c None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:21.171856 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9595a9-9d6b-4f26-bbca-ded4a72c526c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:21.172044 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9595a9-9d6b-4f26-bbca-ded4a72c526c None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:21.173539 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 489/990] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:21.178649 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce5ae8-0c41-484d-b9b7-adadfe7bb663 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:21.179077 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce5ae8-0c41-484d-b9b7-adadfe7bb663 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:21.179340 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce5ae8-0c41-484d-b9b7-adadfe7bb663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:21.195726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-4bce5ae8-0c41-484d-b9b7-adadfe7bb663 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:21.196900 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4bce5ae8-0c41-484d-b9b7-adadfe7bb663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:21.267146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 502/991] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:21.290202 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9595a9-9d6b-4f26-bbca-ded4a72c526c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:21.391534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9595a9-9d6b-4f26-bbca-ded4a72c526c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:21.425334 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20532928-bd8e-4c0d-be91-ab1890e52ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:21.426131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20532928-bd8e-4c0d-be91-ab1890e52ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:21.426416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20532928-bd8e-4c0d-be91-ab1890e52ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:21.426692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20532928-bd8e-4c0d-be91-ab1890e52ac0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:21.427368 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20532928-bd8e-4c0d-be91-ab1890e52ac0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:21.428739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20532928-bd8e-4c0d-be91-ab1890e52ac0 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:21.441161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-20532928-bd8e-4c0d-be91-ab1890e52ac0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 18:50:21.441161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:21.441161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:50:21.441161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:50:21.441161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:50:21.441161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:50:21.441161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:50:21.441161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:21.441161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 18:50:21.441161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:21.441161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:21.441161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:21.441161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:21.441161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:21.441161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:21.441161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:21.441161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:21.447389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:21.447389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:21.447389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:21.447389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:21.447389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:21.447389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:21.447389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:50:21.447389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:50:21.447389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:50:21.447389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:50:21.447389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:21.447389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:21.447389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:21.447389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:21.447389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:21.447389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:21.448253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:21.448253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:21.448253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:21.448253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:21.448253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:21.448253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:21.448253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:21.448253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:21.448253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:21.448253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:21.448253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:50:21.448253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:21.448253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 18:50:21.448253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:21.448253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 490/992] 10.222.0.69 () {56 vars in 1126 bytes} [Thu Jul 14 18:50:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:21.449052 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce5ae8-0c41-484d-b9b7-adadfe7bb663 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:21.463066 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:21.545851 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce5ae8-0c41-484d-b9b7-adadfe7bb663 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:21.579110 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd099f-999f-4165-ada5-03e448126b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:21.579803 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd099f-999f-4165-ada5-03e448126b96 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:21.580648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd099f-999f-4165-ada5-03e448126b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:21.580925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd099f-999f-4165-ada5-03e448126b96 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:21.581632 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd099f-999f-4165-ada5-03e448126b96 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:21.581939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd099f-999f-4165-ada5-03e448126b96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:21.583336 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd099f-999f-4165-ada5-03e448126b96 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:21.593010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 503/993] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 14 18:50:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:21.594217 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20532928-bd8e-4c0d-be91-ab1890e52ac0 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:21.636590 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:21.700626 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20532928-bd8e-4c0d-be91-ab1890e52ac0 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:21.737070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9afea-1193-488b-8d69-067fdac3d945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:21.737742 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9afea-1193-488b-8d69-067fdac3d945 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:21.738026 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9afea-1193-488b-8d69-067fdac3d945 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:21.738303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9afea-1193-488b-8d69-067fdac3d945 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:21.739000 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9afea-1193-488b-8d69-067fdac3d945 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:21.739332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9afea-1193-488b-8d69-067fdac3d945 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:21.740692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9afea-1193-488b-8d69-067fdac3d945 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:21.750196 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 491/994] 10.222.0.69 () {56 vars in 1122 bytes} [Thu Jul 14 18:50:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:21.751368 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd099f-999f-4165-ada5-03e448126b96 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:21.832262 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:21.856102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd099f-999f-4165-ada5-03e448126b96 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:21.890816 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae8890-a276-4864-8bec-3ab308ebabf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:21.891556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae8890-a276-4864-8bec-3ab308ebabf9 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:21.891846 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae8890-a276-4864-8bec-3ab308ebabf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:21.892122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae8890-a276-4864-8bec-3ab308ebabf9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:21.893046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae8890-a276-4864-8bec-3ab308ebabf9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:21.894750 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae8890-a276-4864-8bec-3ab308ebabf9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:21.967193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 504/995] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 14 18:50:21 2022] POST /identity/v3/endpoints => generated 325 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:50:22.160355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d1d00-f1b5-4b9c-9a12-947e420ce89f None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:22.160666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d1d00-f1b5-4b9c-9a12-947e420ce89f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:22.160848 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d1d00-f1b5-4b9c-9a12-947e420ce89f None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:22.162367 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 492/996] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:22.167347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ab16b-ea7e-4659-b4e5-3f4ab5821742 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:22.167643 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ab16b-ea7e-4659-b4e5-3f4ab5821742 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:22.167921 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ab16b-ea7e-4659-b4e5-3f4ab5821742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:22.185161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3c0ab16b-ea7e-4659-b4e5-3f4ab5821742 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:22.186371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3c0ab16b-ea7e-4659-b4e5-3f4ab5821742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:22.261239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 505/997] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:22 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:22.283750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d1d00-f1b5-4b9c-9a12-947e420ce89f None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:22.386217 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d1d00-f1b5-4b9c-9a12-947e420ce89f None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:22.388616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:22.420882 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d48e5-1803-4d99-b110-2f5207cffc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:22.421654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d48e5-1803-4d99-b110-2f5207cffc65 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:22.421944 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d48e5-1803-4d99-b110-2f5207cffc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:22.422225 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d48e5-1803-4d99-b110-2f5207cffc65 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:22.422869 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d48e5-1803-4d99-b110-2f5207cffc65 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:22.424860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d48e5-1803-4d99-b110-2f5207cffc65 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:22.437968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-870d48e5-1803-4d99-b110-2f5207cffc65 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 18:50:22.437968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:22.437968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:50:22.437968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:50:22.437968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:50:22.437968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:50:22.437968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:50:22.437968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:22.437968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 18:50:22.437968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:22.437968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:22.437968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:22.437968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:22.437968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:22.437968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:22.437968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:22.437968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:22.440543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:22.440543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:22.440543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:22.440543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:22.440543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:22.440543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:22.440543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:50:22.440543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:50:22.440543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:50:22.440543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:50:22.440543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:22.440543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:22.440543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:22.440543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:22.440543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:22.440543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:22.441375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:22.441375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:22.441375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:22.441375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:22.441375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:22.441375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:22.441375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:22.441375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:22.441375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:22.441375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:22.441375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:50:22.441375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:22.441375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 18:50:22.441375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:22.441375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 493/998] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:50:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:22.444893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ab16b-ea7e-4659-b4e5-3f4ab5821742 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:22.506521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:22.545837 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ab16b-ea7e-4659-b4e5-3f4ab5821742 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:22.581141 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1e85d-46a4-4674-a66b-8ca2781575ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:22.581816 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1e85d-46a4-4674-a66b-8ca2781575ef None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:22.582106 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1e85d-46a4-4674-a66b-8ca2781575ef None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:22.582386 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1e85d-46a4-4674-a66b-8ca2781575ef None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:22.583093 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1e85d-46a4-4674-a66b-8ca2781575ef None admin] RBAC: Adding query filter params (name=compute) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:22.583437 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1e85d-46a4-4674-a66b-8ca2781575ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:22.584818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1e85d-46a4-4674-a66b-8ca2781575ef None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:22.595376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 506/999] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 14 18:50:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:22.597271 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d48e5-1803-4d99-b110-2f5207cffc65 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:22.623601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb107ce-8079-43e1-a4d4-c3d7cd15f8b7 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:22.623898 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb107ce-8079-43e1-a4d4-c3d7cd15f8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:22.624190 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb107ce-8079-43e1-a4d4-c3d7cd15f8b7 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:22.625680 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 507/1000] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:50:22.630483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294b20d-a74a-4ac7-a33b-985a3b735be4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:22.630777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294b20d-a74a-4ac7-a33b-985a3b735be4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:22.631053 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294b20d-a74a-4ac7-a33b-985a3b735be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:22.649301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c294b20d-a74a-4ac7-a33b-985a3b735be4 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:22.650609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c294b20d-a74a-4ac7-a33b-985a3b735be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:22.707204 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d48e5-1803-4d99-b110-2f5207cffc65 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:22.727589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 508/1001] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:22 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:22.745436 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4cd26f-fa79-4a17-94b3-c84b2af3eabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:22.746103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cd26f-fa79-4a17-94b3-c84b2af3eabc None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:22.746394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cd26f-fa79-4a17-94b3-c84b2af3eabc None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:22.746675 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cd26f-fa79-4a17-94b3-c84b2af3eabc None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:22.747497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4cd26f-fa79-4a17-94b3-c84b2af3eabc None admin] RBAC: Adding query filter params (type=compute) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:22.747820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4cd26f-fa79-4a17-94b3-c84b2af3eabc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:22.748670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294b20d-a74a-4ac7-a33b-985a3b735be4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:22.749193 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4cd26f-fa79-4a17-94b3-c84b2af3eabc None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:22.755680 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 494/1002] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 14 18:50:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:22.761570 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4cd26f-fa79-4a17-94b3-c84b2af3eabc None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:22.797557 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:22.809422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:22.858065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294b20d-a74a-4ac7-a33b-985a3b735be4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:22.874551 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4cd26f-fa79-4a17-94b3-c84b2af3eabc None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:22.893939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d8a97-c383-49ce-9bca-1e739457df22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:22.895175 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d8a97-c383-49ce-9bca-1e739457df22 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:22.895594 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d8a97-c383-49ce-9bca-1e739457df22 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:22.895877 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d8a97-c383-49ce-9bca-1e739457df22 None admin] PATH_INFO: `/v3/services/image` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:22.896527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d8a97-c383-49ce-9bca-1e739457df22 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:22.898429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d8a97-c383-49ce-9bca-1e739457df22 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:22.912158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22636d-f67f-4b4b-8bd4-c5a2c45485f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:22.912816 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22636d-f67f-4b4b-8bd4-c5a2c45485f8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:22.913103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22636d-f67f-4b4b-8bd4-c5a2c45485f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:22.913381 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22636d-f67f-4b4b-8bd4-c5a2c45485f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:22.913636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-ad1d8a97-c383-49ce-9bca-1e739457df22 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 18:50:22.913636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:22.913636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:50:22.913636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:50:22.913636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:50:22.913636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:50:22.913636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:50:22.913636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:22.913636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 18:50:22.913636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:22.913636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:22.913636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:22.913636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:22.913636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:22.913636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:22.913636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:22.913636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:22.914592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:22.914592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:22.914592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:22.914592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:22.914592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:22.914592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:22.914592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:50:22.914592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:50:22.914592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:50:22.914592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:50:22.914592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:22.914592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:22.914592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:22.914592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:22.914592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:22.914592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:22.915472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:22.915472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:22.915472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:22.915472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:22.915472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:22.915472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:22.915472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:22.915472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:22.915472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:22.915472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:22.915472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:50:22.915472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:22.915472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 18:50:22.915472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:22.916294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22636d-f67f-4b4b-8bd4-c5a2c45485f8 None admin] RBAC: Adding query filter params (interface=public, service_id=aeee42a2a1d646f0be30b98d82ea710d, region_id=RegionOne) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:22.916294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22636d-f67f-4b4b-8bd4-c5a2c45485f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:22.916294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22636d-f67f-4b4b-8bd4-c5a2c45485f8 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:22.917110 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 509/1003] 10.222.0.69 () {56 vars in 1105 bytes} [Thu Jul 14 18:50:22 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:22.920594 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d8a97-c383-49ce-9bca-1e739457df22 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:22.923557 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 495/1004] 10.222.0.69 () {56 vars in 1251 bytes} [Thu Jul 14 18:50:22 2022] GET /identity/v3/endpoints?service_id=aeee42a2a1d646f0be30b98d82ea710d&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:22.928931 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22636d-f67f-4b4b-8bd4-c5a2c45485f8 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:23.029993 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:23.036590 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22636d-f67f-4b4b-8bd4-c5a2c45485f8 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:23.051779 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d8a97-c383-49ce-9bca-1e739457df22 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:23.073835 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:23.074289 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc660d-1262-4b11-b8ec-67f650366930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:23.074974 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc660d-1262-4b11-b8ec-67f650366930 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:23.075298 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc660d-1262-4b11-b8ec-67f650366930 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:23.075607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc660d-1262-4b11-b8ec-67f650366930 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:23.076644 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc660d-1262-4b11-b8ec-67f650366930 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:23.078024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc660d-1262-4b11-b8ec-67f650366930 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:23.095207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 496/1005] 10.222.0.69 () {56 vars in 1087 bytes} [Thu Jul 14 18:50:22 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 18:50:23.095465 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34ff75-e360-436e-89a2-619456c39965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:23.095465 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34ff75-e360-436e-89a2-619456c39965 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:23.095465 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34ff75-e360-436e-89a2-619456c39965 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:23.095465 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34ff75-e360-436e-89a2-619456c39965 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:23.095871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34ff75-e360-436e-89a2-619456c39965 None admin] RBAC: Adding query filter params (name=image) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:23.095871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34ff75-e360-436e-89a2-619456c39965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:23.095871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34ff75-e360-436e-89a2-619456c39965 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:23.103771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 510/1006] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:23.104746 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc660d-1262-4b11-b8ec-67f650366930 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:23.216050 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc660d-1262-4b11-b8ec-67f650366930 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:23.254616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a344c13-8045-4229-b73e-5bc3b190cd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:23.255809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a344c13-8045-4229-b73e-5bc3b190cd43 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:23.256040 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a344c13-8045-4229-b73e-5bc3b190cd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:23.256320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a344c13-8045-4229-b73e-5bc3b190cd43 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:23.257029 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a344c13-8045-4229-b73e-5bc3b190cd43 None admin] RBAC: Adding query filter params (type=image) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:23.257340 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a344c13-8045-4229-b73e-5bc3b190cd43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:23.258769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a344c13-8045-4229-b73e-5bc3b190cd43 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:23.263808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 497/1007] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:23 2022] GET /identity/v3/services?type=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:24.115131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626262d3-6297-49f1-8c93-e422b19eb2fb None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:24.115702 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626262d3-6297-49f1-8c93-e422b19eb2fb None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:24.115812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626262d3-6297-49f1-8c93-e422b19eb2fb None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:24.119208 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 511/1008] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:24.122925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f6cf4-08c5-4492-be03-497e7c448a70 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:24.123387 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f6cf4-08c5-4492-be03-497e7c448a70 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:24.123574 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f6cf4-08c5-4492-be03-497e7c448a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:24.145519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a39f6cf4-08c5-4492-be03-497e7c448a70 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:24.146970 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a39f6cf4-08c5-4492-be03-497e7c448a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:24.165541 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:24.216338 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 498/1009] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:24 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:24.240144 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626262d3-6297-49f1-8c93-e422b19eb2fb None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:24.343530 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626262d3-6297-49f1-8c93-e422b19eb2fb None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:24.376050 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689b5d1-6696-474e-8535-2ac5813e7caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:24.377373 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689b5d1-6696-474e-8535-2ac5813e7caa None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:24.377656 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689b5d1-6696-474e-8535-2ac5813e7caa None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:24.377927 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689b5d1-6696-474e-8535-2ac5813e7caa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:24.379098 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689b5d1-6696-474e-8535-2ac5813e7caa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:24.380558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689b5d1-6696-474e-8535-2ac5813e7caa None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:24.382339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 512/1010] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:24.387838 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f6cf4-08c5-4492-be03-497e7c448a70 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:24.503786 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f6cf4-08c5-4492-be03-497e7c448a70 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:24.540278 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34401890-c7ca-4e9e-81c5-ac7e3618a951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:24.540943 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34401890-c7ca-4e9e-81c5-ac7e3618a951 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:24.541230 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34401890-c7ca-4e9e-81c5-ac7e3618a951 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:24.541506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34401890-c7ca-4e9e-81c5-ac7e3618a951 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:24.542228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34401890-c7ca-4e9e-81c5-ac7e3618a951 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:24.543700 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34401890-c7ca-4e9e-81c5-ac7e3618a951 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:24.628585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 499/1011] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 14 18:50:24 2022] POST /identity/v3/projects => generated 311 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:50:25.158962 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26dd89-e182-46cf-8a1c-ded22c51059f None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:25.159539 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26dd89-e182-46cf-8a1c-ded22c51059f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:25.159539 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26dd89-e182-46cf-8a1c-ded22c51059f None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:25.161063 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 513/1012] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:25.166097 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d732e-0aac-459f-8535-1eae015c00c7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:25.166468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d732e-0aac-459f-8535-1eae015c00c7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:25.166666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d732e-0aac-459f-8535-1eae015c00c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:25.178952 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:25.184504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c93d732e-0aac-459f-8535-1eae015c00c7 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:25.185694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c93d732e-0aac-459f-8535-1eae015c00c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:25.254234 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 500/1013] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:25 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:25.273813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26dd89-e182-46cf-8a1c-ded22c51059f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:25.330996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:25.380573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26dd89-e182-46cf-8a1c-ded22c51059f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:25.389139 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008155a-a1db-440c-a33b-50edc547deeb None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:25.389437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008155a-a1db-440c-a33b-50edc547deeb None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:25.389926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008155a-a1db-440c-a33b-50edc547deeb None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:25.391299 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 501/1014] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:50:25.396114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046864f8-397e-4357-b484-7347a6d80468 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:25.396399 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046864f8-397e-4357-b484-7347a6d80468 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:25.396687 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046864f8-397e-4357-b484-7347a6d80468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:25.415105 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-046864f8-397e-4357-b484-7347a6d80468 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:25.416331 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-046864f8-397e-4357-b484-7347a6d80468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:25.416940 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6b2eb-b5d4-4871-a309-590b731e8670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:25.417747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6b2eb-b5d4-4871-a309-590b731e8670 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:25.418041 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6b2eb-b5d4-4871-a309-590b731e8670 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:25.418315 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6b2eb-b5d4-4871-a309-590b731e8670 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:25.418955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad6b2eb-b5d4-4871-a309-590b731e8670 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:25.420374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad6b2eb-b5d4-4871-a309-590b731e8670 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:25.432823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-7ad6b2eb-b5d4-4871-a309-590b731e8670 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 18:50:25.432823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:25.432823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:50:25.432823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:50:25.432823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:50:25.432823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:50:25.432823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:50:25.432823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:25.432823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 18:50:25.432823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:25.432823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:25.432823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:25.432823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:25.432823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:25.432823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:25.432823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:25.432823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:25.433888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:25.433888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:25.433888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:25.433888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:25.433888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:25.433888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:25.433888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:50:25.433888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:50:25.433888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:50:25.433888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:50:25.433888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:25.433888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:25.433888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:25.433888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:25.433888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:25.433888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:25.434735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:25.434735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:25.434735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:25.434735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:25.434735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:25.434735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:25.434735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:25.434735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:25.434735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:25.434735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:25.434735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:50:25.434735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:25.434735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 18:50:25.434735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:25.434735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 514/1015] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:50:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:25.440964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6b2eb-b5d4-4871-a309-590b731e8670 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:25.493616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 502/1016] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:25 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:25.516101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046864f8-397e-4357-b484-7347a6d80468 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:25.553952 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6b2eb-b5d4-4871-a309-590b731e8670 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:25.570894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:25.582767 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:25.590123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20d388-6802-41d9-a645-30d25b9712a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:25.591029 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20d388-6802-41d9-a645-30d25b9712a9 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:25.591550 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20d388-6802-41d9-a645-30d25b9712a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:25.592012 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20d388-6802-41d9-a645-30d25b9712a9 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:25.592877 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20d388-6802-41d9-a645-30d25b9712a9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:25.593337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20d388-6802-41d9-a645-30d25b9712a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:25.594873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20d388-6802-41d9-a645-30d25b9712a9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:25.600723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 515/1017] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 14 18:50:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:25.606242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20d388-6802-41d9-a645-30d25b9712a9 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:25.623211 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046864f8-397e-4357-b484-7347a6d80468 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:25.660682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad0d3f-0f0e-49b9-a20c-461364902d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:25.661352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad0d3f-0f0e-49b9-a20c-461364902d03 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:25.661640 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad0d3f-0f0e-49b9-a20c-461364902d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:25.661911 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad0d3f-0f0e-49b9-a20c-461364902d03 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:25.662636 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad0d3f-0f0e-49b9-a20c-461364902d03 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:25.664060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad0d3f-0f0e-49b9-a20c-461364902d03 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:25.714738 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20d388-6802-41d9-a645-30d25b9712a9 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:25.755207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 503/1018] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 14 18:50:25 2022] POST /identity/v3/services => generated 242 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:50:25.765560 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f6342-6e2b-4595-b2f8-0acc8995f0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:25.766247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f6342-6e2b-4595-b2f8-0acc8995f0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:25.766529 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f6342-6e2b-4595-b2f8-0acc8995f0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:25.766804 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f6342-6e2b-4595-b2f8-0acc8995f0e2 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:25.768238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f6342-6e2b-4595-b2f8-0acc8995f0e2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:25.768604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f6342-6e2b-4595-b2f8-0acc8995f0e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:25.770067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f6342-6e2b-4595-b2f8-0acc8995f0e2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:25.778045 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 516/1019] 10.222.0.69 () {56 vars in 1112 bytes} [Thu Jul 14 18:50:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:25.787319 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad0d3f-0f0e-49b9-a20c-461364902d03 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:25.901415 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad0d3f-0f0e-49b9-a20c-461364902d03 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:25.929582 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:25.940098 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba474212-cbf8-4aa6-9714-041b88ddafdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:25.940840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba474212-cbf8-4aa6-9714-041b88ddafdb None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:25.941176 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba474212-cbf8-4aa6-9714-041b88ddafdb None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:25.941470 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba474212-cbf8-4aa6-9714-041b88ddafdb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:25.942166 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba474212-cbf8-4aa6-9714-041b88ddafdb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:25.943576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba474212-cbf8-4aa6-9714-041b88ddafdb None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:26.022431 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 504/1020] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 14 18:50:25 2022] POST /identity/v3/endpoints => generated 334 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:50:26.700675 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4634f-6cf0-4cfc-b10e-59a82cb651c0 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:26.701193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4634f-6cf0-4cfc-b10e-59a82cb651c0 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:26.701297 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4634f-6cf0-4cfc-b10e-59a82cb651c0 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:26.702750 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 517/1021] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:26.708715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cae077-a1c1-45e3-8072-f977950d802e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:26.709163 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cae077-a1c1-45e3-8072-f977950d802e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:26.709275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cae077-a1c1-45e3-8072-f977950d802e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:26.726617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-81cae077-a1c1-45e3-8072-f977950d802e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:26.728111 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-81cae077-a1c1-45e3-8072-f977950d802e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:26.803494 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 505/1022] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:26.827468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b4634f-6cf0-4cfc-b10e-59a82cb651c0 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:26.872933 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:26.934246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b4634f-6cf0-4cfc-b10e-59a82cb651c0 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:26.967707 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4508c-2ba0-44a3-bb47-66b34ec3767e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:26.968437 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4508c-2ba0-44a3-bb47-66b34ec3767e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:26.968723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4508c-2ba0-44a3-bb47-66b34ec3767e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:26.969005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4508c-2ba0-44a3-bb47-66b34ec3767e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:26.970185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4508c-2ba0-44a3-bb47-66b34ec3767e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:26.972329 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4508c-2ba0-44a3-bb47-66b34ec3767e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:26.974174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 518/1023] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:26.979830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cae077-a1c1-45e3-8072-f977950d802e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:27.086793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cae077-a1c1-45e3-8072-f977950d802e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:27.096752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:27.121559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a01a5f-6b64-4a34-ae94-28c9d090e4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:27.122232 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a01a5f-6b64-4a34-ae94-28c9d090e4e8 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:27.122517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a01a5f-6b64-4a34-ae94-28c9d090e4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:27.122793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a01a5f-6b64-4a34-ae94-28c9d090e4e8 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:27.123538 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a01a5f-6b64-4a34-ae94-28c9d090e4e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:27.124972 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a01a5f-6b64-4a34-ae94-28c9d090e4e8 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:27.239273 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 506/1024] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 14 18:50:26 2022] POST /identity/v3/users => generated 285 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:50:27.824512 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfc789-bafd-4b73-95e9-6f681b55b897 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:27.825032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfc789-bafd-4b73-95e9-6f681b55b897 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:27.825481 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfc789-bafd-4b73-95e9-6f681b55b897 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:27.827990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 519/1025] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:27.833360 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da545a96-72b9-4ffe-9537-f677ccbe7145 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:27.833811 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da545a96-72b9-4ffe-9537-f677ccbe7145 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:27.834378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da545a96-72b9-4ffe-9537-f677ccbe7145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:27.851422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-da545a96-72b9-4ffe-9537-f677ccbe7145 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:27.852614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-da545a96-72b9-4ffe-9537-f677ccbe7145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:27.922118 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 507/1026] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:27.945320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfc789-bafd-4b73-95e9-6f681b55b897 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:28.059089 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfc789-bafd-4b73-95e9-6f681b55b897 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:28.068796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:28.093232 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87833030-12ad-4eac-ba86-b8e891e3f225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:28.093959 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87833030-12ad-4eac-ba86-b8e891e3f225 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:28.094246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87833030-12ad-4eac-ba86-b8e891e3f225 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:28.094531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87833030-12ad-4eac-ba86-b8e891e3f225 None admin] PATH_INFO: `/v3/services/image` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:28.095198 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87833030-12ad-4eac-ba86-b8e891e3f225 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:28.097267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87833030-12ad-4eac-ba86-b8e891e3f225 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:28.099565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbf746-89ea-403c-929d-c491d9ae0cf1 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:28.099909 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbf746-89ea-403c-929d-c491d9ae0cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:28.100200 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbf746-89ea-403c-929d-c491d9ae0cf1 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:28.101564 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 508/1027] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:28.106449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4dd6e-c218-473d-bb4c-d6d630b0e94c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:28.106745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4dd6e-c218-473d-bb4c-d6d630b0e94c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:28.107032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4dd6e-c218-473d-bb4c-d6d630b0e94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:28.109593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-87833030-12ad-4eac-ba86-b8e891e3f225 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 18:50:28.109593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:28.109593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:50:28.109593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:50:28.109593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:50:28.109593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:50:28.109593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:50:28.109593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:28.109593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 18:50:28.109593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:28.109593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:28.109593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:28.109593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:28.109593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:28.109593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:28.109593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:28.109593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:28.110606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:28.110606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:28.110606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:28.110606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:28.110606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:28.110606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:28.110606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:50:28.110606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:50:28.110606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:50:28.110606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:50:28.110606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:28.110606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:28.110606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:28.110606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:28.110606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:28.110606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:28.117019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:28.117019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:28.117019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:28.117019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:28.117019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:28.117019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:28.117019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:28.117019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:28.117019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:28.117019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:28.117019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:50:28.117019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:28.117019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 18:50:28.117019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:28.117019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 520/1028] 10.222.0.69 () {56 vars in 1105 bytes} [Thu Jul 14 18:50:27 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:28.119664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87833030-12ad-4eac-ba86-b8e891e3f225 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:28.129729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-6ba4dd6e-c218-473d-bb4c-d6d630b0e94c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:28.131044 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6ba4dd6e-c218-473d-bb4c-d6d630b0e94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:28.201907 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 509/1029] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:28.223743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87833030-12ad-4eac-ba86-b8e891e3f225 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:28.224877 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4dd6e-c218-473d-bb4c-d6d630b0e94c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:28.250188 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:28.258707 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44ce74-1851-4bb8-8325-916fb848f012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:28.259396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44ce74-1851-4bb8-8325-916fb848f012 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:28.259677 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44ce74-1851-4bb8-8325-916fb848f012 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:28.260764 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44ce74-1851-4bb8-8325-916fb848f012 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:28.260764 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44ce74-1851-4bb8-8325-916fb848f012 None admin] RBAC: Adding query filter params (name=image) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:28.260967 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44ce74-1851-4bb8-8325-916fb848f012 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:28.262289 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44ce74-1851-4bb8-8325-916fb848f012 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:28.267183 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 521/1030] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:28.273641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44ce74-1851-4bb8-8325-916fb848f012 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:28.312396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:28.331333 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4dd6e-c218-473d-bb4c-d6d630b0e94c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:28.367832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e2a0e-d893-486f-98cb-7ebf9f9e3497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:28.368566 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e2a0e-d893-486f-98cb-7ebf9f9e3497 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:28.368851 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e2a0e-d893-486f-98cb-7ebf9f9e3497 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:28.369127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e2a0e-d893-486f-98cb-7ebf9f9e3497 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:28.378936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44ce74-1851-4bb8-8325-916fb848f012 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:28.380030 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20e2a0e-d893-486f-98cb-7ebf9f9e3497 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:50:28.388819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-a20e2a0e-d893-486f-98cb-7ebf9f9e3497 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 18:50:28.388819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:28.388819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:28.388819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:28.388819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:28.388819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:28.388819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:28.388819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:28.388819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:28.388819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:28.388819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:28.388819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:28.388819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:50:28.388819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:50:28.388819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:50:28.388819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:50:28.389769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:50:28.389769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:50:28.389769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:50:28.389769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:50:28.389769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:28.389769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:28.389769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:28.389769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:28.389769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:28.389769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:28.389769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:28.389769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:28.389769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:28.389769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:28.389769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:28.389769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:28.390744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:28.390744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:28.390744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:28.390744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:28.390744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:50:28.390744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:50:28.390744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:50:28.390744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:50:28.390744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:50:28.390744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:50:28.390744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 18:50:28.390744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:28.390744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 510/1031] 10.222.0.69 () {56 vars in 1120 bytes} [Thu Jul 14 18:50:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:28.395972 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e2a0e-d893-486f-98cb-7ebf9f9e3497 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:28.417891 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf84864-d3cf-4c0b-8b14-2211050871c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:28.418567 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf84864-d3cf-4c0b-8b14-2211050871c2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:28.418859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf84864-d3cf-4c0b-8b14-2211050871c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:28.419137 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf84864-d3cf-4c0b-8b14-2211050871c2 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:28.419881 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf84864-d3cf-4c0b-8b14-2211050871c2 None admin] RBAC: Adding query filter params (type=image) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:28.420197 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf84864-d3cf-4c0b-8b14-2211050871c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:28.421580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf84864-d3cf-4c0b-8b14-2211050871c2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:28.432656 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 522/1032] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:28.434336 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf84864-d3cf-4c0b-8b14-2211050871c2 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:28.488792 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:28.504147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e2a0e-d893-486f-98cb-7ebf9f9e3497 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:28.540830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a5e23-c7c8-4b32-91a0-1a3773b21776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:28.541770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a5e23-c7c8-4b32-91a0-1a3773b21776 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:28.542341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a5e23-c7c8-4b32-91a0-1a3773b21776 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:28.542806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a5e23-c7c8-4b32-91a0-1a3773b21776 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:28.543228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:28.544575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a5e23-c7c8-4b32-91a0-1a3773b21776 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:28.545054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a5e23-c7c8-4b32-91a0-1a3773b21776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:28.546143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf84864-d3cf-4c0b-8b14-2211050871c2 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:28.546763 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a5e23-c7c8-4b32-91a0-1a3773b21776 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:28.556496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 511/1033] 10.222.0.69 () {56 vars in 1115 bytes} [Thu Jul 14 18:50:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:28.561743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a5e23-c7c8-4b32-91a0-1a3773b21776 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:28.586390 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5efeab-4633-464b-9537-7caee6cc7a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:28.587051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5efeab-4633-464b-9537-7caee6cc7a76 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:28.587367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5efeab-4633-464b-9537-7caee6cc7a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:28.587648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5efeab-4633-464b-9537-7caee6cc7a76 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:28.588436 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5efeab-4633-464b-9537-7caee6cc7a76 None admin] RBAC: Adding query filter params (interface=public, service_id=8581260fbf3045e9b2c5b2c49412b215, region_id=RegionOne) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:28.588740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5efeab-4633-464b-9537-7caee6cc7a76 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:28.590108 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5efeab-4633-464b-9537-7caee6cc7a76 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:28.599188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 523/1034] 10.222.0.69 () {56 vars in 1251 bytes} [Thu Jul 14 18:50:28 2022] GET /identity/v3/endpoints?service_id=8581260fbf3045e9b2c5b2c49412b215&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:28.601838 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5efeab-4633-464b-9537-7caee6cc7a76 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:28.674914 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a5e23-c7c8-4b32-91a0-1a3773b21776 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:28.703410 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5efeab-4633-464b-9537-7caee6cc7a76 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:28.713625 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc600dc7-027e-4e2c-956e-9de378ff32d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:28.714419 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc600dc7-027e-4e2c-956e-9de378ff32d4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:28.714706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc600dc7-027e-4e2c-956e-9de378ff32d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:28.714990 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc600dc7-027e-4e2c-956e-9de378ff32d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:28.726712 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc600dc7-027e-4e2c-956e-9de378ff32d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:28.728204 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc600dc7-027e-4e2c-956e-9de378ff32d4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:28.735138 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:28.741971 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52137d57-ffba-4edf-b919-b2e45a166c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:28.742862 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52137d57-ffba-4edf-b919-b2e45a166c60 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:28.743361 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52137d57-ffba-4edf-b919-b2e45a166c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:28.743841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52137d57-ffba-4edf-b919-b2e45a166c60 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:28.744710 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52137d57-ffba-4edf-b919-b2e45a166c60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:28.746259 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52137d57-ffba-4edf-b919-b2e45a166c60 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:28.746468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-cc600dc7-027e-4e2c-956e-9de378ff32d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:28.746468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:28.746468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:28.746468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:28.746468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:28.746468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:28.746468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:28.746468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:28.746468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:28.746468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:28.746468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:28.746468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:28.746468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:28.746468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:28.746468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:28.751052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:28.751052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:28.751052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:28.751052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:28.751052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:28.751052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:28.751052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:28.751052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:28.751052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:28.751052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:28.751052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:28.751052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:28.751052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:28.751052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:28.751052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:28.751052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:28.751052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:28.752205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:28.752205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:28.752205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:28.752205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:28.752205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 512/1035] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:28.753291 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc600dc7-027e-4e2c-956e-9de378ff32d4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:28.756301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 524/1036] 10.222.0.69 () {56 vars in 1087 bytes} [Thu Jul 14 18:50:28 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 18:50:28.863995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc600dc7-027e-4e2c-956e-9de378ff32d4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:28.900848 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7535a11-41bc-4b2a-a984-dc221f4c49f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:28.901529 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7535a11-41bc-4b2a-a984-dc221f4c49f2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:28.901820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7535a11-41bc-4b2a-a984-dc221f4c49f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:28.902978 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7535a11-41bc-4b2a-a984-dc221f4c49f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:28.903085 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7535a11-41bc-4b2a-a984-dc221f4c49f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:28.903436 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7535a11-41bc-4b2a-a984-dc221f4c49f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:28.904824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7535a11-41bc-4b2a-a984-dc221f4c49f2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:28.918280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 513/1037] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:28.924813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52137d57-ffba-4edf-b919-b2e45a166c60 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:28.958762 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:29.035217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52137d57-ffba-4edf-b919-b2e45a166c60 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:29.070462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086a4e8-8d92-4c8a-a0e2-5673091f2970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:29.071311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086a4e8-8d92-4c8a-a0e2-5673091f2970 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:29.071603 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086a4e8-8d92-4c8a-a0e2-5673091f2970 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:29.071882 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086a4e8-8d92-4c8a-a0e2-5673091f2970 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:29.079209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086a4e8-8d92-4c8a-a0e2-5673091f2970 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:29.081327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086a4e8-8d92-4c8a-a0e2-5673091f2970 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:29.095873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-3086a4e8-8d92-4c8a-a0e2-5673091f2970 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 18:50:29.095873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:29.095873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:29.095873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:29.095873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:29.095873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:29.095873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:29.095873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:29.095873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:29.095873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:29.095873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:29.095873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:29.095873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:50:29.095873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:50:29.095873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:50:29.095873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:50:29.096907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:29.096907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:29.096907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:50:29.096907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:29.096907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:50:29.096907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:29.096907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:29.096907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:29.096907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:29.096907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:29.096907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:29.096907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:29.096907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:29.096907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:29.096907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:29.096907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:29.096907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:29.096907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:29.097855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:29.097855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:29.097855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:50:29.097855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:50:29.097855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:50:29.097855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:50:29.097855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:50:29.097855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:50:29.097855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 18:50:29.097855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:29.099236 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 525/1038] 10.222.0.69 () {56 vars in 1093 bytes} [Thu Jul 14 18:50:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:29.103057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7535a11-41bc-4b2a-a984-dc221f4c49f2 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:29.171482 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:29.214322 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7535a11-41bc-4b2a-a984-dc221f4c49f2 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:29.249963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea09c17-9ec5-4e30-b484-4253f2d18d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:29.250639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea09c17-9ec5-4e30-b484-4253f2d18d59 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:29.250928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea09c17-9ec5-4e30-b484-4253f2d18d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:29.251231 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea09c17-9ec5-4e30-b484-4253f2d18d59 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:29.252688 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea09c17-9ec5-4e30-b484-4253f2d18d59 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:29.253003 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea09c17-9ec5-4e30-b484-4253f2d18d59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:29.254436 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea09c17-9ec5-4e30-b484-4253f2d18d59 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:29.271752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 514/1039] 10.222.0.69 () {56 vars in 1133 bytes} [Thu Jul 14 18:50:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:29.277816 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086a4e8-8d92-4c8a-a0e2-5673091f2970 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:29.364158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:29.375899 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086a4e8-8d92-4c8a-a0e2-5673091f2970 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:29.378718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c08dd1-2000-4b58-b04c-3508de3ba979 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:29.379013 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c08dd1-2000-4b58-b04c-3508de3ba979 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:29.379326 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c08dd1-2000-4b58-b04c-3508de3ba979 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:29.380695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 515/1040] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:29.385602 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c114759-6efe-41e2-8bfd-c77b5124fa8c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:29.385889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c114759-6efe-41e2-8bfd-c77b5124fa8c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:29.386171 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c114759-6efe-41e2-8bfd-c77b5124fa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:29.403065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-9c114759-6efe-41e2-8bfd-c77b5124fa8c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:29.404280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9c114759-6efe-41e2-8bfd-c77b5124fa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:29.415218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29292842-c562-4966-a590-06d2bdcb08f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:29.415959 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29292842-c562-4966-a590-06d2bdcb08f8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:29.416159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29292842-c562-4966-a590-06d2bdcb08f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:29.416438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29292842-c562-4966-a590-06d2bdcb08f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:29.475110 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 516/1041] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:29.488501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29292842-c562-4966-a590-06d2bdcb08f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:29.490027 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29292842-c562-4966-a590-06d2bdcb08f8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:29.498376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c114759-6efe-41e2-8bfd-c77b5124fa8c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:29.547760 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:29.583689 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:29.597985 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 526/1042] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:50:29 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:50:29.603666 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29292842-c562-4966-a590-06d2bdcb08f8 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:29.604621 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c114759-6efe-41e2-8bfd-c77b5124fa8c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:29.644211 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a27d6-b2a1-4529-8a92-5dcbe3d4d9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:29.645017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a27d6-b2a1-4529-8a92-5dcbe3d4d9bb None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:29.645330 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a27d6-b2a1-4529-8a92-5dcbe3d4d9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:29.645615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a27d6-b2a1-4529-8a92-5dcbe3d4d9bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:29.656499 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a27d6-b2a1-4529-8a92-5dcbe3d4d9bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:50:29.664682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-4c3a27d6-b2a1-4529-8a92-5dcbe3d4d9bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:50:29.664682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:29.664682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:29.664682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:29.664682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:29.664682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:29.664682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:29.664682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:29.664682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:29.664682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:29.664682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:29.664682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:29.664682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:50:29.664682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:50:29.664682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:50:29.664682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:50:29.665617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:50:29.665617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:50:29.665617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:50:29.665617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:50:29.665617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:29.665617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:29.665617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:29.665617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:29.665617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:29.665617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:29.665617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:29.665617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:29.665617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:29.665617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:29.665617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:29.665617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:29.666458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:29.666458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:29.666458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:29.666458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:29.666458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:50:29.666458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:50:29.666458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:50:29.666458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:50:29.666458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:50:29.666458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:50:29.666458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:50:29.666458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:29.666458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 517/1043] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:50:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:29.671774 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a27d6-b2a1-4529-8a92-5dcbe3d4d9bb None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:29.712118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29292842-c562-4966-a590-06d2bdcb08f8 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:29.750322 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891776bf-146e-497d-838c-0c3d62bdd95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:29.751047 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891776bf-146e-497d-838c-0c3d62bdd95a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:29.751362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891776bf-146e-497d-838c-0c3d62bdd95a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:29.751669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891776bf-146e-497d-838c-0c3d62bdd95a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:29.762245 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891776bf-146e-497d-838c-0c3d62bdd95a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:29.763760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891776bf-146e-497d-838c-0c3d62bdd95a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:29.779415 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a27d6-b2a1-4529-8a92-5dcbe3d4d9bb None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:29.781482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-891776bf-146e-497d-838c-0c3d62bdd95a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:29.781482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:29.781482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:29.781482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:29.781482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:29.781482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:29.781482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:29.781482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:29.781482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:29.781482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:29.781482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:29.781482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:29.781482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:29.781482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:29.781482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:29.782352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:29.782352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:29.782352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:29.782352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:29.782352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:29.782352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:29.782352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:29.782352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:29.782352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:29.782352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:29.782352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:29.782352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:29.782352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:29.782352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:29.782352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:29.782352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:29.782352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:29.783799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:29.783799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:29.783799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:29.783799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:29.783799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 527/1044] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:29.788895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891776bf-146e-497d-838c-0c3d62bdd95a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:29.789777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:29.819658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31660c8-3aaf-4ba4-80c7-47866bf6ea2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:29.820335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31660c8-3aaf-4ba4-80c7-47866bf6ea2a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:29.820623 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31660c8-3aaf-4ba4-80c7-47866bf6ea2a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:29.820905 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31660c8-3aaf-4ba4-80c7-47866bf6ea2a None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:29.821654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31660c8-3aaf-4ba4-80c7-47866bf6ea2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:29.822157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31660c8-3aaf-4ba4-80c7-47866bf6ea2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:29.823356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31660c8-3aaf-4ba4-80c7-47866bf6ea2a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:29.836764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 518/1045] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:50:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:29.837839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31660c8-3aaf-4ba4-80c7-47866bf6ea2a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:29.839996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:29.907388 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891776bf-146e-497d-838c-0c3d62bdd95a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:29.945185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12523640-93a8-4df2-b66b-c6bcc7123b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:29.945876 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12523640-93a8-4df2-b66b-c6bcc7123b87 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:29.946140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12523640-93a8-4df2-b66b-c6bcc7123b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:29.946421 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12523640-93a8-4df2-b66b-c6bcc7123b87 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:29.947138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12523640-93a8-4df2-b66b-c6bcc7123b87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:29.947488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12523640-93a8-4df2-b66b-c6bcc7123b87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:29.948865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12523640-93a8-4df2-b66b-c6bcc7123b87 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:29.952515 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31660c8-3aaf-4ba4-80c7-47866bf6ea2a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:29.961053 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 528/1046] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:29.966564 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12523640-93a8-4df2-b66b-c6bcc7123b87 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:29.996441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6921ae9-a56c-4d43-9ec8-ec524e683b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:29.997325 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6921ae9-a56c-4d43-9ec8-ec524e683b6b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:29.997612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6921ae9-a56c-4d43-9ec8-ec524e683b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:29.997889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6921ae9-a56c-4d43-9ec8-ec524e683b6b None admin] PATH_INFO: `/v3/users/cdd9fe183ab645739fb6ac71f698758c` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:30.012241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6921ae9-a56c-4d43-9ec8-ec524e683b6b None admin] RBAC: Authorizing `identity:get_user(user_id=cdd9fe183ab645739fb6ac71f698758c)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:30.013702 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6921ae9-a56c-4d43-9ec8-ec524e683b6b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:30.015622 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 519/1047] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:50:29 2022] GET /identity/v3/users/cdd9fe183ab645739fb6ac71f698758c => generated 285 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:30.021612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6921ae9-a56c-4d43-9ec8-ec524e683b6b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:30.049730 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:30.074328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:30.077311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12523640-93a8-4df2-b66b-c6bcc7123b87 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:30.112190 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec924e40-325d-458d-bb72-2966f1307dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:30.112994 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec924e40-325d-458d-bb72-2966f1307dad None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:30.113289 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec924e40-325d-458d-bb72-2966f1307dad None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:30.113572 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec924e40-325d-458d-bb72-2966f1307dad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:30.120259 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec924e40-325d-458d-bb72-2966f1307dad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:30.121760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec924e40-325d-458d-bb72-2966f1307dad None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:30.126560 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6921ae9-a56c-4d43-9ec8-ec524e683b6b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:30.136358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-ec924e40-325d-458d-bb72-2966f1307dad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:30.136358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:30.136358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:30.136358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:30.136358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:30.136358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:30.136358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:30.136358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:30.136358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:30.136358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:30.136358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:30.136358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:30.136358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:50:30.136358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:50:30.136358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:50:30.137237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:50:30.137237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:30.137237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:30.137237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:30.137237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:30.137237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:30.137237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:30.137237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:30.137237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:30.137237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:30.137237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:30.137237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:30.137237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:30.137237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:30.137237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:30.137237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:30.137237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:30.138893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:50:30.138893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:50:30.138893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:50:30.138893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:50:30.138893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:50:30.138893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:50:30.138893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:30.138893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:30.138893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 529/1048] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:50:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:30.144004 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec924e40-325d-458d-bb72-2966f1307dad None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:30.164968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e33f5-0782-4602-bf58-222e28585794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:30.165675 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e33f5-0782-4602-bf58-222e28585794 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:30.165968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e33f5-0782-4602-bf58-222e28585794 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:30.166253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e33f5-0782-4602-bf58-222e28585794 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:30.239987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e33f5-0782-4602-bf58-222e28585794 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:30.241450 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e33f5-0782-4602-bf58-222e28585794 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:30.250480 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec924e40-325d-458d-bb72-2966f1307dad None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:30.267867 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:30.289236 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0fa032-bf5f-41dd-9fac-db295485c0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:30.289908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0fa032-bf5f-41dd-9fac-db295485c0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:30.290196 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0fa032-bf5f-41dd-9fac-db295485c0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:30.290500 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0fa032-bf5f-41dd-9fac-db295485c0b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:30.291293 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0fa032-bf5f-41dd-9fac-db295485c0b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:30.291612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0fa032-bf5f-41dd-9fac-db295485c0b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:30.293061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0fa032-bf5f-41dd-9fac-db295485c0b8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:30.311499 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 530/1049] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:50:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:30.312612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0fa032-bf5f-41dd-9fac-db295485c0b8 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:30.335681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:30.347246 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 520/1050] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:50:30 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 18:50:30.353242 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e33f5-0782-4602-bf58-222e28585794 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:30.422572 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0fa032-bf5f-41dd-9fac-db295485c0b8 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:30.461950 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b3ddc-0834-4dcd-a4d1-33be53346ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:30.462616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b3ddc-0834-4dcd-a4d1-33be53346ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:30.462903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b3ddc-0834-4dcd-a4d1-33be53346ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:30.463195 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b3ddc-0834-4dcd-a4d1-33be53346ca1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:30.464026 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b3ddc-0834-4dcd-a4d1-33be53346ca1 None admin] RBAC: Adding query filter params (role.id=4def7095cc264443ab935a01966235a0, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=8d27d6df5cb64697adebd961dfbec324) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:30.464340 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b3ddc-0834-4dcd-a4d1-33be53346ca1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:30.465341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e33f5-0782-4602-bf58-222e28585794 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:30.465782 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b3ddc-0834-4dcd-a4d1-33be53346ca1 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:30.479370 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 531/1051] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:50:30 2022] GET /identity/v3/role_assignments?user.id=8d27d6df5cb64697adebd961dfbec324&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=4def7095cc264443ab935a01966235a0 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:30.502782 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:30.512203 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52cafd-1800-4633-baf6-99a055c3f90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:30.513092 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52cafd-1800-4633-baf6-99a055c3f90f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:30.513382 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52cafd-1800-4633-baf6-99a055c3f90f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:30.513665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52cafd-1800-4633-baf6-99a055c3f90f None admin] PATH_INFO: `/v3/projects/25b5df177a964556af065fb2b1f1ca46` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:30.514880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52cafd-1800-4633-baf6-99a055c3f90f None admin] RBAC: Authorizing `identity:get_project(project_id=25b5df177a964556af065fb2b1f1ca46)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:30.516380 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52cafd-1800-4633-baf6-99a055c3f90f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:30.525663 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 521/1052] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:50:30 2022] GET /identity/v3/projects/25b5df177a964556af065fb2b1f1ca46 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:30.525844 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b3ddc-0834-4dcd-a4d1-33be53346ca1 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:30.616185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:30.634054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b3ddc-0834-4dcd-a4d1-33be53346ca1 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:30.671562 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468819ee-91dc-4fb8-92d9-3c8775b1a2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:30.672228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468819ee-91dc-4fb8-92d9-3c8775b1a2dc None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:30.672517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468819ee-91dc-4fb8-92d9-3c8775b1a2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:30.672797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468819ee-91dc-4fb8-92d9-3c8775b1a2dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:30.674155 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468819ee-91dc-4fb8-92d9-3c8775b1a2dc None admin] RBAC: Adding query filter params (role.id=27fc14631c494ca0b2dc7f708be016c2, scope.project.id=25b5df177a964556af065fb2b1f1ca46, user.id=cdd9fe183ab645739fb6ac71f698758c) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:30.674471 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468819ee-91dc-4fb8-92d9-3c8775b1a2dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:30.675951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468819ee-91dc-4fb8-92d9-3c8775b1a2dc None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:30.681883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 532/1053] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:50:30 2022] GET /identity/v3/role_assignments?user.id=cdd9fe183ab645739fb6ac71f698758c&scope.project.id=25b5df177a964556af065fb2b1f1ca46&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:30.764779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcde28-8014-4eaa-b3e1-e2869fdbd575 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:30.765128 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcde28-8014-4eaa-b3e1-e2869fdbd575 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:30.765341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcde28-8014-4eaa-b3e1-e2869fdbd575 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:30.766865 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 522/1054] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:30.772957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45d6b0-dd84-4b03-a4b9-d944e2af221f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:30.773275 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45d6b0-dd84-4b03-a4b9-d944e2af221f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:30.773534 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45d6b0-dd84-4b03-a4b9-d944e2af221f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:30.792085 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7a45d6b0-dd84-4b03-a4b9-d944e2af221f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:30.793229 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7a45d6b0-dd84-4b03-a4b9-d944e2af221f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:30.867204 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 533/1055] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:30.887745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcde28-8014-4eaa-b3e1-e2869fdbd575 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:30.992333 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcde28-8014-4eaa-b3e1-e2869fdbd575 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:31.034285 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece05a4-32e6-4267-a655-89a42a4ea7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:31.035080 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece05a4-32e6-4267-a655-89a42a4ea7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:31.035403 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece05a4-32e6-4267-a655-89a42a4ea7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:31.035682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece05a4-32e6-4267-a655-89a42a4ea7c4 None admin] PATH_INFO: `/v3/services/image` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:31.036331 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece05a4-32e6-4267-a655-89a42a4ea7c4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:31.037704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece05a4-32e6-4267-a655-89a42a4ea7c4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:31.049480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-1ece05a4-32e6-4267-a655-89a42a4ea7c4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 18:50:31.049480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:31.049480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:50:31.049480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:50:31.049480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:50:31.049480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:50:31.049480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:50:31.049480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:31.049480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 18:50:31.049480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:31.049480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:50:31.049480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:31.049480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:31.049480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:31.049480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:31.049480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:31.049480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:31.050428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:31.050428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:31.050428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:31.050428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:31.050428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:31.050428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:31.050428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:50:31.050428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:50:31.050428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:50:31.050428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:50:31.050428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:31.050428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:31.050428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:31.050428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:31.050428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:31.050428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:31.051292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:31.051292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:31.051292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:31.051292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:31.051292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:31.051292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:31.051292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:31.051292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:31.051292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:31.051292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:31.051292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:50:31.051292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:50:31.051292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 18:50:31.051292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:31.051292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 523/1056] 10.222.0.69 () {56 vars in 1105 bytes} [Thu Jul 14 18:50:30 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:31.058629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45d6b0-dd84-4b03-a4b9-d944e2af221f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:31.141013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:31.163155 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45d6b0-dd84-4b03-a4b9-d944e2af221f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:31.197474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef834abd-0b83-4109-a8bb-07afcea3823e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:31.198144 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef834abd-0b83-4109-a8bb-07afcea3823e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:31.198426 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef834abd-0b83-4109-a8bb-07afcea3823e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:31.198698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef834abd-0b83-4109-a8bb-07afcea3823e None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:31.199429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef834abd-0b83-4109-a8bb-07afcea3823e None admin] RBAC: Adding query filter params (name=image) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:31.199727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef834abd-0b83-4109-a8bb-07afcea3823e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:31.201093 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef834abd-0b83-4109-a8bb-07afcea3823e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:31.211229 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 534/1057] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:31.212765 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece05a4-32e6-4267-a655-89a42a4ea7c4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:31.273390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:31.315552 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece05a4-32e6-4267-a655-89a42a4ea7c4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:31.349169 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d265a-f1b7-49f7-bb6a-e0a3cdd1bdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:31.349833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d265a-f1b7-49f7-bb6a-e0a3cdd1bdc8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:31.350121 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d265a-f1b7-49f7-bb6a-e0a3cdd1bdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:31.350402 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d265a-f1b7-49f7-bb6a-e0a3cdd1bdc8 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:31.351109 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d265a-f1b7-49f7-bb6a-e0a3cdd1bdc8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:31.351450 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d265a-f1b7-49f7-bb6a-e0a3cdd1bdc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:31.352831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d265a-f1b7-49f7-bb6a-e0a3cdd1bdc8 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:31.362600 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 524/1058] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:31.364061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef834abd-0b83-4109-a8bb-07afcea3823e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:31.465703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef834abd-0b83-4109-a8bb-07afcea3823e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:31.501221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce534e54-d8ee-46fe-8ee7-89ec34db2e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:31.502421 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce534e54-d8ee-46fe-8ee7-89ec34db2e26 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:31.502711 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce534e54-d8ee-46fe-8ee7-89ec34db2e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:31.502989 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce534e54-d8ee-46fe-8ee7-89ec34db2e26 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:31.503764 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce534e54-d8ee-46fe-8ee7-89ec34db2e26 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:31.505143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce534e54-d8ee-46fe-8ee7-89ec34db2e26 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:31.574277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:31.581117 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 535/1059] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 14 18:50:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:50:32.510427 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a69eb-4303-432f-bfc4-22b806547082 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:32.510983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a69eb-4303-432f-bfc4-22b806547082 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:32.510983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a69eb-4303-432f-bfc4-22b806547082 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:32.512438 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 525/1060] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:32.518139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc3af2-1933-40c0-bb61-db4f84b8f7d8 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:32.518541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc3af2-1933-40c0-bb61-db4f84b8f7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:32.518756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc3af2-1933-40c0-bb61-db4f84b8f7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:32.536325 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6cbc3af2-1933-40c0-bb61-db4f84b8f7d8 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:32.537523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6cbc3af2-1933-40c0-bb61-db4f84b8f7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:32.612654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 536/1061] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:32.634632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a69eb-4303-432f-bfc4-22b806547082 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:32.684207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50983cda-efcf-48b6-9cae-87c56f32e5da None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:32.684524 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50983cda-efcf-48b6-9cae-87c56f32e5da None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:32.684779 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50983cda-efcf-48b6-9cae-87c56f32e5da None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:32.686173 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 537/1062] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:50:32.691805 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e183cbc-76ea-4fab-b8fb-47a190f13862 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:32.692148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e183cbc-76ea-4fab-b8fb-47a190f13862 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:32.692380 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e183cbc-76ea-4fab-b8fb-47a190f13862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:32.709687 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2e183cbc-76ea-4fab-b8fb-47a190f13862 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:32.710897 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2e183cbc-76ea-4fab-b8fb-47a190f13862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:32.746402 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a69eb-4303-432f-bfc4-22b806547082 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:32.770046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:32.780514 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 538/1063] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:32.784186 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce9e00-70b1-4792-aa2c-2ac6665232d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:32.785007 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce9e00-70b1-4792-aa2c-2ac6665232d4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:32.785291 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce9e00-70b1-4792-aa2c-2ac6665232d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:32.785566 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce9e00-70b1-4792-aa2c-2ac6665232d4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:32.795788 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce9e00-70b1-4792-aa2c-2ac6665232d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:50:32.803887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e183cbc-76ea-4fab-b8fb-47a190f13862 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:32.804789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-00ce9e00-70b1-4792-aa2c-2ac6665232d4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 18:50:32.804789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:32.804789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:32.804789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:32.804789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:32.804789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:32.804789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:32.804789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:32.804789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:32.804789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:32.804789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:32.804789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:32.804789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:50:32.804789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:50:32.804789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:50:32.804789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:50:32.805706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:50:32.805706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:50:32.805706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:50:32.805706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:50:32.805706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:32.805706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:32.805706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:32.805706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:32.805706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:32.805706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:32.805706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:32.805706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:32.805706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:32.805706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:32.805706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:32.805706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:32.806611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:32.806611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:32.806611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:32.806611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:32.806611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:50:32.806611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:50:32.806611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:50:32.806611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:50:32.806611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:50:32.806611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:50:32.806611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 18:50:32.806611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:32.806611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 526/1064] 10.222.0.69 () {56 vars in 1120 bytes} [Thu Jul 14 18:50:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:32.812120 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce9e00-70b1-4792-aa2c-2ac6665232d4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:32.866268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:32.912057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e183cbc-76ea-4fab-b8fb-47a190f13862 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:32.912967 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce9e00-70b1-4792-aa2c-2ac6665232d4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:32.947221 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be38df-a6a2-4d77-9b32-4bfae8112f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:32.947426 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3748a30-783c-4dd2-9e6a-ba7dd61e012d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:32.947900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be38df-a6a2-4d77-9b32-4bfae8112f6c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:32.948068 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3748a30-783c-4dd2-9e6a-ba7dd61e012d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:32.948222 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be38df-a6a2-4d77-9b32-4bfae8112f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:32.948314 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3748a30-783c-4dd2-9e6a-ba7dd61e012d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:32.948471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be38df-a6a2-4d77-9b32-4bfae8112f6c None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:32.948562 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3748a30-783c-4dd2-9e6a-ba7dd61e012d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:32.949203 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be38df-a6a2-4d77-9b32-4bfae8112f6c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:32.949515 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be38df-a6a2-4d77-9b32-4bfae8112f6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:32.950884 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be38df-a6a2-4d77-9b32-4bfae8112f6c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:32.963513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 527/1065] 10.222.0.69 () {56 vars in 1115 bytes} [Thu Jul 14 18:50:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:32.963682 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3748a30-783c-4dd2-9e6a-ba7dd61e012d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:50:32.964506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be38df-a6a2-4d77-9b32-4bfae8112f6c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:32.966907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-e3748a30-783c-4dd2-9e6a-ba7dd61e012d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:50:32.966907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:32.966907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:32.966907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:32.966907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:32.966907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:32.966907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:32.966907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:32.966907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:32.966907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:32.966907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:32.966907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:32.966907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:50:32.966907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:50:32.966907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:50:32.966907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:50:32.968049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:50:32.968049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:50:32.968049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:50:32.968049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:50:32.968049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:32.968049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:32.968049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:32.968049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:32.968049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:32.968049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:32.968049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:32.968049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:32.968049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:32.968049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:32.968049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:32.968049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:32.968908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:32.968908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:32.968908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:32.968908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:32.968908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:50:32.968908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:50:32.968908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:50:32.968908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:50:32.968908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:50:32.968908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:50:32.968908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:50:32.968908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:32.968908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 539/1066] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:50:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:32.973888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3748a30-783c-4dd2-9e6a-ba7dd61e012d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:33.022580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:33.081386 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be38df-a6a2-4d77-9b32-4bfae8112f6c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:33.088797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3748a30-783c-4dd2-9e6a-ba7dd61e012d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:33.103761 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:33.124570 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb02ace-510d-4e4d-a16c-4b8d9032254f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:33.125368 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb02ace-510d-4e4d-a16c-4b8d9032254f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:33.125650 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb02ace-510d-4e4d-a16c-4b8d9032254f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:33.125923 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb02ace-510d-4e4d-a16c-4b8d9032254f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:33.127763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f16fb1-6b9f-4222-9a65-215612e20813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:33.128673 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f16fb1-6b9f-4222-9a65-215612e20813 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:33.128983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f16fb1-6b9f-4222-9a65-215612e20813 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:33.129352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f16fb1-6b9f-4222-9a65-215612e20813 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:33.130107 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f16fb1-6b9f-4222-9a65-215612e20813 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:33.130421 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f16fb1-6b9f-4222-9a65-215612e20813 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:33.131836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f16fb1-6b9f-4222-9a65-215612e20813 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:33.137595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb02ace-510d-4e4d-a16c-4b8d9032254f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:33.139025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb02ace-510d-4e4d-a16c-4b8d9032254f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:33.147135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 540/1067] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:50:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:33.148178 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f16fb1-6b9f-4222-9a65-215612e20813 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:33.159195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-ddb02ace-510d-4e4d-a16c-4b8d9032254f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:33.159195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:33.159195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:33.159195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:33.159195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:33.159195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:33.159195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:33.159195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:33.159195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:33.159195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:33.159195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:33.159195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:33.159195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:33.159195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:33.159195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:33.160102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:33.160102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:33.160102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:33.160102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:33.160102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:33.160102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:33.160102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:33.160102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:33.160102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:33.160102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:33.160102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:33.160102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:33.160102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:33.160102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:33.160102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:33.160102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:33.160102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:33.161011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:33.161011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:33.161011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:33.161011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:33.161011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 528/1068] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:33.166118 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb02ace-510d-4e4d-a16c-4b8d9032254f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:33.261484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f16fb1-6b9f-4222-9a65-215612e20813 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:33.268451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:33.274540 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb02ace-510d-4e4d-a16c-4b8d9032254f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:33.301323 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3b594-7ef1-437d-9295-3a3d31ed5c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:33.302648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3b594-7ef1-437d-9295-3a3d31ed5c16 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:33.303432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3b594-7ef1-437d-9295-3a3d31ed5c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:33.304141 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3b594-7ef1-437d-9295-3a3d31ed5c16 None admin] PATH_INFO: `/v3/users/cdd9fe183ab645739fb6ac71f698758c` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:33.305680 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3b594-7ef1-437d-9295-3a3d31ed5c16 None admin] RBAC: Authorizing `identity:get_user(user_id=cdd9fe183ab645739fb6ac71f698758c)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:33.307879 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3b594-7ef1-437d-9295-3a3d31ed5c16 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:33.310046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 541/1069] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:50:33 2022] GET /identity/v3/users/cdd9fe183ab645739fb6ac71f698758c => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:33.315055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20780f-1f63-426a-b29a-bd52422688b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:33.315353 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3b594-7ef1-437d-9295-3a3d31ed5c16 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:33.315756 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20780f-1f63-426a-b29a-bd52422688b7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:33.316044 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20780f-1f63-426a-b29a-bd52422688b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:33.316325 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20780f-1f63-426a-b29a-bd52422688b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:33.317052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20780f-1f63-426a-b29a-bd52422688b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:33.317362 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20780f-1f63-426a-b29a-bd52422688b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:33.318725 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20780f-1f63-426a-b29a-bd52422688b7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:33.335648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 529/1070] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:33.336503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20780f-1f63-426a-b29a-bd52422688b7 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:33.361850 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:33.421482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3b594-7ef1-437d-9295-3a3d31ed5c16 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:33.446912 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20780f-1f63-426a-b29a-bd52422688b7 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:33.459339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a3405-591f-463d-8ef5-d8ef801f621f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:33.460213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a3405-591f-463d-8ef5-d8ef801f621f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:33.460506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a3405-591f-463d-8ef5-d8ef801f621f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:33.460789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a3405-591f-463d-8ef5-d8ef801f621f None admin] PATH_INFO: `/v3/projects/25b5df177a964556af065fb2b1f1ca46` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:33.461992 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a3405-591f-463d-8ef5-d8ef801f621f None admin] RBAC: Authorizing `identity:get_project(project_id=25b5df177a964556af065fb2b1f1ca46)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:33.463478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a3405-591f-463d-8ef5-d8ef801f621f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:33.469909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 542/1071] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:50:33 2022] GET /identity/v3/projects/25b5df177a964556af065fb2b1f1ca46 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:33.470940 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a3405-591f-463d-8ef5-d8ef801f621f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:33.488991 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645864d-03a8-498e-88c9-fd0a440f9a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:33.489794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645864d-03a8-498e-88c9-fd0a440f9a3e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:33.490088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645864d-03a8-498e-88c9-fd0a440f9a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:33.490372 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645864d-03a8-498e-88c9-fd0a440f9a3e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:33.498304 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645864d-03a8-498e-88c9-fd0a440f9a3e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:33.499895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645864d-03a8-498e-88c9-fd0a440f9a3e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:33.514714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-a645864d-03a8-498e-88c9-fd0a440f9a3e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 18:50:33.514714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:33.514714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:33.514714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:33.514714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:33.514714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:33.514714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:33.514714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:33.514714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:33.514714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:33.514714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:33.514714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:33.514714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:50:33.514714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:50:33.514714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:50:33.514714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:50:33.516131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:33.516131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:33.516131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:50:33.516131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:33.516131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:50:33.516131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:33.516131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:33.516131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:33.516131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:33.516131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:33.516131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:33.516131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:33.516131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:33.516131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:33.516131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:33.516131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:33.516131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:33.516131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:33.517055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:33.517055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:33.517055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:50:33.517055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:50:33.517055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:50:33.517055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:50:33.517055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:50:33.517055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:50:33.517055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 18:50:33.517055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:33.517055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 530/1072] 10.222.0.69 () {56 vars in 1093 bytes} [Thu Jul 14 18:50:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:33.522178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645864d-03a8-498e-88c9-fd0a440f9a3e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:33.542525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:33.580689 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a3405-591f-463d-8ef5-d8ef801f621f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:33.595538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:33.616720 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6ee89-6c97-49b9-bd82-728881000ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:33.617620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6ee89-6c97-49b9-bd82-728881000ad0 None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:33.617910 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6ee89-6c97-49b9-bd82-728881000ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:33.618191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6ee89-6c97-49b9-bd82-728881000ad0 None admin] PATH_INFO: `/v3/projects/25b5df177a964556af065fb2b1f1ca46/users/cdd9fe183ab645739fb6ac71f698758c/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:33.620257 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6ee89-6c97-49b9-bd82-728881000ad0 None admin] RBAC: Authorizing `identity:create_grant(project_id=25b5df177a964556af065fb2b1f1ca46, user_id=cdd9fe183ab645739fb6ac71f698758c, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:33.621710 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6ee89-6c97-49b9-bd82-728881000ad0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:33.629520 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645864d-03a8-498e-88c9-fd0a440f9a3e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:33.663464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b6dc5-c8c1-4df1-9bad-ded492eb3de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:33.664138 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b6dc5-c8c1-4df1-9bad-ded492eb3de3 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:33.664425 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b6dc5-c8c1-4df1-9bad-ded492eb3de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:33.664710 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b6dc5-c8c1-4df1-9bad-ded492eb3de3 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:33.665543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b6dc5-c8c1-4df1-9bad-ded492eb3de3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:33.665859 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b6dc5-c8c1-4df1-9bad-ded492eb3de3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:33.667313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b6dc5-c8c1-4df1-9bad-ded492eb3de3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:33.684531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 531/1073] 10.222.0.69 () {56 vars in 1133 bytes} [Thu Jul 14 18:50:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:33.685807 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa1791-e55e-4426-882c-405c71008a4a None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:33.686118 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa1791-e55e-4426-882c-405c71008a4a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:33.686412 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa1791-e55e-4426-882c-405c71008a4a None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:33.687678 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 532/1074] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:33.690707 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa1791-e55e-4426-882c-405c71008a4a None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:33.722343 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 543/1075] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:50:33 2022] PUT /identity/v3/projects/25b5df177a964556af065fb2b1f1ca46/users/cdd9fe183ab645739fb6ac71f698758c/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 18:50:33.723692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf2040-912d-42ac-87cd-e8e5c7f83054 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:33.723987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf2040-912d-42ac-87cd-e8e5c7f83054 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:33.724266 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf2040-912d-42ac-87cd-e8e5c7f83054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:33.748607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7cdf2040-912d-42ac-87cd-e8e5c7f83054 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:33.749854 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7cdf2040-912d-42ac-87cd-e8e5c7f83054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:33.780082 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:33.810905 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa1791-e55e-4426-882c-405c71008a4a None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:33.830393 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 544/1076] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:33.848933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f551f7-29e6-422e-93c4-d76e6b36bb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:33.849750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f551f7-29e6-422e-93c4-d76e6b36bb19 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:33.850041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f551f7-29e6-422e-93c4-d76e6b36bb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:33.850325 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f551f7-29e6-422e-93c4-d76e6b36bb19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:33.853051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf2040-912d-42ac-87cd-e8e5c7f83054 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:33.862226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f551f7-29e6-422e-93c4-d76e6b36bb19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:33.863752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f551f7-29e6-422e-93c4-d76e6b36bb19 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:33.885158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-67f551f7-29e6-422e-93c4-d76e6b36bb19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:33.885158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:33.885158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:33.885158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:33.885158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:33.885158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:33.885158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:33.885158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:33.885158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:33.885158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:33.885158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:33.885158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:33.885158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:33.885158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:33.885158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:33.886106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:33.886106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:33.886106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:33.886106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:33.886106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:33.886106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:33.886106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:33.886106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:33.886106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:33.886106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:33.886106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:33.886106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:33.886106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:33.886106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:33.886106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:33.886106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:33.886106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:33.886995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:33.886995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:33.886995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:33.886995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:33.886995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 533/1077] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:33.892365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f551f7-29e6-422e-93c4-d76e6b36bb19 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:33.965157 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf2040-912d-42ac-87cd-e8e5c7f83054 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:33.966747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:34.007189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f419aba-0908-48d3-8855-764a78273307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:34.007427 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f551f7-29e6-422e-93c4-d76e6b36bb19 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:34.007861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f419aba-0908-48d3-8855-764a78273307 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:34.008147 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f419aba-0908-48d3-8855-764a78273307 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:34.008425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f419aba-0908-48d3-8855-764a78273307 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:34.035637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:34.045633 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4f148-48bc-41f6-8a5d-cac2e0ace96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:34.046310 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4f148-48bc-41f6-8a5d-cac2e0ace96f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:34.046600 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4f148-48bc-41f6-8a5d-cac2e0ace96f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:34.046895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4f148-48bc-41f6-8a5d-cac2e0ace96f None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:34.047779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4f148-48bc-41f6-8a5d-cac2e0ace96f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:34.048098 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4f148-48bc-41f6-8a5d-cac2e0ace96f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:34.050499 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4f148-48bc-41f6-8a5d-cac2e0ace96f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:34.067064 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 534/1078] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:34.068226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4f148-48bc-41f6-8a5d-cac2e0ace96f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:34.080125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f419aba-0908-48d3-8855-764a78273307 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:34.081583 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f419aba-0908-48d3-8855-764a78273307 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:34.175569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4f148-48bc-41f6-8a5d-cac2e0ace96f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:34.186884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 545/1079] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:50:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:50:34.192477 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f419aba-0908-48d3-8855-764a78273307 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:34.195308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:34.212684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b382d-771d-4b10-8db4-71804620ac3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:34.213487 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b382d-771d-4b10-8db4-71804620ac3c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:34.213776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b382d-771d-4b10-8db4-71804620ac3c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:34.214059 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b382d-771d-4b10-8db4-71804620ac3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:34.221757 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b382d-771d-4b10-8db4-71804620ac3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:34.223229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b382d-771d-4b10-8db4-71804620ac3c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:34.238522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-5a9b382d-771d-4b10-8db4-71804620ac3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:34.238522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:34.238522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:34.238522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:34.238522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:34.238522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:34.238522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:34.238522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:34.238522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:34.238522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:34.238522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:34.238522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:34.238522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:50:34.238522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:50:34.238522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:50:34.239573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:50:34.239573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:34.239573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:34.239573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:34.239573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:34.239573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:34.239573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:34.239573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:34.239573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:34.239573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:34.239573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:34.239573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:34.239573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:34.239573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:34.239573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:34.239573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:34.239573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:34.240445 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:50:34.240445 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:50:34.240445 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:50:34.240445 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:50:34.240445 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:50:34.240445 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:50:34.240445 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:34.240445 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:34.244781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 535/1080] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:50:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:34.245621 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b382d-771d-4b10-8db4-71804620ac3c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:34.296465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:34.303994 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f419aba-0908-48d3-8855-764a78273307 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:34.338786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c112aea-b236-410d-ad3e-1e49d11c6db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:34.339625 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c112aea-b236-410d-ad3e-1e49d11c6db4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:34.339913 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c112aea-b236-410d-ad3e-1e49d11c6db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:34.340190 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c112aea-b236-410d-ad3e-1e49d11c6db4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:34.351583 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c112aea-b236-410d-ad3e-1e49d11c6db4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:34.353040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c112aea-b236-410d-ad3e-1e49d11c6db4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:34.355234 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b382d-771d-4b10-8db4-71804620ac3c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:34.370949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-8c112aea-b236-410d-ad3e-1e49d11c6db4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:34.370949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:34.370949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:34.370949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:34.370949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:34.370949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:34.370949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:34.370949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:34.370949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:34.370949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:34.370949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:34.370949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:34.370949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:34.370949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:34.370949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:34.371929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:34.371929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:34.371929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:34.371929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:34.371929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:34.371929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:34.371929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:34.371929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:34.371929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:34.371929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:34.371929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:34.371929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:34.371929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:34.371929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:34.371929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:34.371929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:34.371929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:34.372816 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:34.372816 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:34.372816 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:34.372816 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:34.372816 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 546/1081] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:34.378044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c112aea-b236-410d-ad3e-1e49d11c6db4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:34.393622 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd633088-b83a-4391-a925-03d8ecb4e65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:34.394561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd633088-b83a-4391-a925-03d8ecb4e65c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:34.395046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd633088-b83a-4391-a925-03d8ecb4e65c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:34.395154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd633088-b83a-4391-a925-03d8ecb4e65c None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:34.395959 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd633088-b83a-4391-a925-03d8ecb4e65c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:34.396234 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd633088-b83a-4391-a925-03d8ecb4e65c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:34.397711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd633088-b83a-4391-a925-03d8ecb4e65c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:34.411195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 536/1082] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:50:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:34.417333 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd633088-b83a-4391-a925-03d8ecb4e65c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:34.438197 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:34.487041 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c112aea-b236-410d-ad3e-1e49d11c6db4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:34.524625 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd633088-b83a-4391-a925-03d8ecb4e65c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:34.525168 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2acaeb-6f63-44f5-8fa5-730835957580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:34.525767 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2acaeb-6f63-44f5-8fa5-730835957580 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:34.526166 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2acaeb-6f63-44f5-8fa5-730835957580 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:34.526442 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2acaeb-6f63-44f5-8fa5-730835957580 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:34.527680 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2acaeb-6f63-44f5-8fa5-730835957580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:34.527878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2acaeb-6f63-44f5-8fa5-730835957580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:34.529218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2acaeb-6f63-44f5-8fa5-730835957580 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:34.533768 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:34.543695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 547/1083] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:34.561694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94402ec-5af4-4ef0-92c4-099d85467a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:34.562620 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94402ec-5af4-4ef0-92c4-099d85467a4e None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:34.562909 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94402ec-5af4-4ef0-92c4-099d85467a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:34.563215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94402ec-5af4-4ef0-92c4-099d85467a4e None admin] PATH_INFO: `/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/8d27d6df5cb64697adebd961dfbec324/roles/4def7095cc264443ab935a01966235a0` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:34.565597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94402ec-5af4-4ef0-92c4-099d85467a4e None admin] RBAC: Authorizing `identity:create_grant(project_id=6003dba8ddd247e7842d28ea7ca5238c, user_id=8d27d6df5cb64697adebd961dfbec324, role_id=4def7095cc264443ab935a01966235a0)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:34.567201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94402ec-5af4-4ef0-92c4-099d85467a4e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:34.643425 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 537/1084] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:50:34 2022] PUT /identity/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/8d27d6df5cb64697adebd961dfbec324/roles/4def7095cc264443ab935a01966235a0 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 18:50:35.736998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3b384-745d-47e1-944a-397aed9d8523 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:35.737621 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3b384-745d-47e1-944a-397aed9d8523 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:35.737621 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3b384-745d-47e1-944a-397aed9d8523 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:35.738973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 548/1085] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:35.744897 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce9b58-245e-4e4e-9567-9d1511d4d845 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:35.745391 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce9b58-245e-4e4e-9567-9d1511d4d845 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:35.745587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce9b58-245e-4e4e-9567-9d1511d4d845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:35.761921 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-60ce9b58-245e-4e4e-9567-9d1511d4d845 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:35.763087 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-60ce9b58-245e-4e4e-9567-9d1511d4d845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:35.828753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 538/1086] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:35.851759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3b384-745d-47e1-944a-397aed9d8523 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:35.951971 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3b384-745d-47e1-944a-397aed9d8523 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:35.965894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:35.985747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a03cf-51ee-44e5-96da-f0de636fa84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:35.986578 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a03cf-51ee-44e5-96da-f0de636fa84b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:35.986863 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a03cf-51ee-44e5-96da-f0de636fa84b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:35.987213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a03cf-51ee-44e5-96da-f0de636fa84b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:35.998638 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a03cf-51ee-44e5-96da-f0de636fa84b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:50:36.006883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-0b8a03cf-51ee-44e5-96da-f0de636fa84b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:50:36.006883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:36.006883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:36.006883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:36.006883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:36.006883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:36.006883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:36.006883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:36.006883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:36.006883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:36.006883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:36.006883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:36.006883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:50:36.006883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:50:36.006883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:50:36.006883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:50:36.007869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:50:36.007869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:50:36.007869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:50:36.007869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:50:36.007869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:36.007869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:36.007869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:36.007869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:36.007869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:36.007869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:36.007869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:36.007869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:36.007869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:36.007869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:36.007869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:36.007869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:36.008794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:36.008794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:36.008794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:36.008794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:36.008794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:50:36.008794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:50:36.008794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:50:36.008794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:50:36.008794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:50:36.008794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:50:36.008794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:50:36.008794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:36.013526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 549/1087] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:50:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:36.014249 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce9b58-245e-4e4e-9567-9d1511d4d845 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:36.106033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:36.111981 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce9b58-245e-4e4e-9567-9d1511d4d845 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:36.145621 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d2819-d122-41b9-920e-8202eb3d80e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:36.146286 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d2819-d122-41b9-920e-8202eb3d80e4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:36.146574 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d2819-d122-41b9-920e-8202eb3d80e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:36.146854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d2819-d122-41b9-920e-8202eb3d80e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:36.147641 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d2819-d122-41b9-920e-8202eb3d80e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:36.147954 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d2819-d122-41b9-920e-8202eb3d80e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:36.149326 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d2819-d122-41b9-920e-8202eb3d80e4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:36.162724 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 539/1088] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:50:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:36.164143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a03cf-51ee-44e5-96da-f0de636fa84b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:36.261649 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a03cf-51ee-44e5-96da-f0de636fa84b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:36.294846 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95e737-0ef9-4eb5-b4da-d8315b515fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:36.295755 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95e737-0ef9-4eb5-b4da-d8315b515fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:36.296043 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95e737-0ef9-4eb5-b4da-d8315b515fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:36.296319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95e737-0ef9-4eb5-b4da-d8315b515fa0 None admin] PATH_INFO: `/v3/users/cdd9fe183ab645739fb6ac71f698758c` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:36.297485 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95e737-0ef9-4eb5-b4da-d8315b515fa0 None admin] RBAC: Authorizing `identity:get_user(user_id=cdd9fe183ab645739fb6ac71f698758c)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:36.298920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95e737-0ef9-4eb5-b4da-d8315b515fa0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:36.300591 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 550/1089] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:50:36 2022] GET /identity/v3/users/cdd9fe183ab645739fb6ac71f698758c => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:36.306256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d2819-d122-41b9-920e-8202eb3d80e4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:36.407241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d2819-d122-41b9-920e-8202eb3d80e4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:36.440683 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c9ef2-baa9-4a78-92ec-d89a8b5804ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:36.441325 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c9ef2-baa9-4a78-92ec-d89a8b5804ff None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:36.441613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c9ef2-baa9-4a78-92ec-d89a8b5804ff None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:36.441891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c9ef2-baa9-4a78-92ec-d89a8b5804ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:36.467552 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:36.508360 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c9ef2-baa9-4a78-92ec-d89a8b5804ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:36.509813 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c9ef2-baa9-4a78-92ec-d89a8b5804ff None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:36.605839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 540/1090] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:50:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:50:36.612177 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95e737-0ef9-4eb5-b4da-d8315b515fa0 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:36.632582 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfd93e-f30d-4b1a-818b-fdcd9333c19d None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:36.632878 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfd93e-f30d-4b1a-818b-fdcd9333c19d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:36.633162 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfd93e-f30d-4b1a-818b-fdcd9333c19d None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:36.634551 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 541/1091] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:50:36.639238 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b2b7a-8eff-4b6b-ba4a-3dd76c3e3e49 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:36.639530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b2b7a-8eff-4b6b-ba4a-3dd76c3e3e49 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:36.639813 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b2b7a-8eff-4b6b-ba4a-3dd76c3e3e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:36.654653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:36.656515 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-843b2b7a-8eff-4b6b-ba4a-3dd76c3e3e49 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:36.658242 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-843b2b7a-8eff-4b6b-ba4a-3dd76c3e3e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:36.710047 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:36.711453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95e737-0ef9-4eb5-b4da-d8315b515fa0 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:36.724189 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 542/1092] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:36.744429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0952f-796e-4966-990f-92398c758a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:36.745289 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0952f-796e-4966-990f-92398c758a1e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:36.745571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0952f-796e-4966-990f-92398c758a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:36.745844 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0952f-796e-4966-990f-92398c758a1e None admin] PATH_INFO: `/v3/projects/25b5df177a964556af065fb2b1f1ca46` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:36.747012 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0952f-796e-4966-990f-92398c758a1e None admin] RBAC: Authorizing `identity:get_project(project_id=25b5df177a964556af065fb2b1f1ca46)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:36.747259 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b2b7a-8eff-4b6b-ba4a-3dd76c3e3e49 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:36.748509 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0952f-796e-4966-990f-92398c758a1e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:36.750207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 551/1093] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:50:36 2022] GET /identity/v3/projects/25b5df177a964556af065fb2b1f1ca46 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:36.756221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0952f-796e-4966-990f-92398c758a1e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:36.844845 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b2b7a-8eff-4b6b-ba4a-3dd76c3e3e49 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:36.853965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0952f-796e-4966-990f-92398c758a1e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:36.863949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:36.877659 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa798d3-86c3-4c37-89d3-83e13b730a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:36.878468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa798d3-86c3-4c37-89d3-83e13b730a0f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:36.878756 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa798d3-86c3-4c37-89d3-83e13b730a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:36.879032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa798d3-86c3-4c37-89d3-83e13b730a0f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:36.885513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49550f5-6663-4ba4-9b4c-64f1eb0e8383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:36.886158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49550f5-6663-4ba4-9b4c-64f1eb0e8383 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:36.886446 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49550f5-6663-4ba4-9b4c-64f1eb0e8383 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:36.886722 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49550f5-6663-4ba4-9b4c-64f1eb0e8383 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:36.904530 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49550f5-6663-4ba4-9b4c-64f1eb0e8383 None admin] RBAC: Adding query filter params (role.id=27fc14631c494ca0b2dc7f708be016c2, scope.project.id=25b5df177a964556af065fb2b1f1ca46, user.id=cdd9fe183ab645739fb6ac71f698758c) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:36.904530 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49550f5-6663-4ba4-9b4c-64f1eb0e8383 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:36.904735 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa798d3-86c3-4c37-89d3-83e13b730a0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:50:36.904813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49550f5-6663-4ba4-9b4c-64f1eb0e8383 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:36.904813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 552/1094] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:50:36 2022] GET /identity/v3/role_assignments?user.id=cdd9fe183ab645739fb6ac71f698758c&scope.project.id=25b5df177a964556af065fb2b1f1ca46&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:50:36.904963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-8aa798d3-86c3-4c37-89d3-83e13b730a0f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 18:50:36.904963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:36.904963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:36.904963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:36.904963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:36.904963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:36.904963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:36.904963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:36.904963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:36.904963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:36.904963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:36.904963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:36.904963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:50:36.904963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:50:36.904963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:50:36.904963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:50:36.905828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:50:36.905828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:50:36.905828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:50:36.905828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:50:36.905828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:36.905828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:36.905828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:36.905828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:36.905828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:36.905828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:36.905828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:36.905828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:36.905828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:36.905828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:36.905828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:36.905828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:36.906684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:36.906684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:36.906684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:36.906684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:36.906684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:50:36.906684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:50:36.906684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:50:36.906684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:50:36.906684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:50:36.906684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:50:36.906684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 18:50:36.906684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:36.906684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 543/1095] 10.222.0.69 () {56 vars in 1120 bytes} [Thu Jul 14 18:50:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:50:36.907448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49550f5-6663-4ba4-9b4c-64f1eb0e8383 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:37.011731 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49550f5-6663-4ba4-9b4c-64f1eb0e8383 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:37.047442 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc178851-c88b-4b0d-a6e8-ac5d3bdede01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:37.048122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc178851-c88b-4b0d-a6e8-ac5d3bdede01 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:37.048419 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc178851-c88b-4b0d-a6e8-ac5d3bdede01 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:37.048705 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc178851-c88b-4b0d-a6e8-ac5d3bdede01 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:37.049447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc178851-c88b-4b0d-a6e8-ac5d3bdede01 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:37.049788 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc178851-c88b-4b0d-a6e8-ac5d3bdede01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:37.051323 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc178851-c88b-4b0d-a6e8-ac5d3bdede01 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:37.059718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 553/1096] 10.222.0.69 () {56 vars in 1115 bytes} [Thu Jul 14 18:50:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:37.065766 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa798d3-86c3-4c37-89d3-83e13b730a0f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:37.116050 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:37.165987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa798d3-86c3-4c37-89d3-83e13b730a0f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:37.203130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8429c-836b-4a97-8b5c-59b42696ac2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:37.203939 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8429c-836b-4a97-8b5c-59b42696ac2e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:37.204222 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8429c-836b-4a97-8b5c-59b42696ac2e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:37.204604 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8429c-836b-4a97-8b5c-59b42696ac2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:37.215495 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8429c-836b-4a97-8b5c-59b42696ac2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:37.216942 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8429c-836b-4a97-8b5c-59b42696ac2e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:37.233632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-95e8429c-836b-4a97-8b5c-59b42696ac2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:37.233632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:37.233632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:37.233632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:37.233632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:37.233632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:37.233632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:37.233632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:37.233632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:37.233632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:37.233632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:37.233632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:37.233632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:37.233632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:37.233632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:37.234738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:37.234738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:37.234738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:37.234738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:37.234738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:37.234738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:37.234738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:37.234738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:37.234738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:37.234738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:37.234738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:37.234738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:37.234738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:37.234738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:37.234738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:37.234738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:37.234738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:37.240477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:37.240477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:37.240477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:37.240477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:37.240477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 544/1097] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:37.242150 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc178851-c88b-4b0d-a6e8-ac5d3bdede01 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:37.277824 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:37.341271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc178851-c88b-4b0d-a6e8-ac5d3bdede01 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:37.374078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7daa3-0dc8-4302-8d09-75e38dec82b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:37.374671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7daa3-0dc8-4302-8d09-75e38dec82b2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:37.374955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7daa3-0dc8-4302-8d09-75e38dec82b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:37.375263 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7daa3-0dc8-4302-8d09-75e38dec82b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:37.375980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7daa3-0dc8-4302-8d09-75e38dec82b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:37.376287 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7daa3-0dc8-4302-8d09-75e38dec82b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:37.378194 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7daa3-0dc8-4302-8d09-75e38dec82b2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:37.389318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 554/1098] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:37.395687 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8429c-836b-4a97-8b5c-59b42696ac2e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:37.495891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:37.498874 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8429c-836b-4a97-8b5c-59b42696ac2e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:37.533320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f1ab2-ab9c-4108-8877-51cb1918c62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:37.534114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f1ab2-ab9c-4108-8877-51cb1918c62a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:37.534402 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f1ab2-ab9c-4108-8877-51cb1918c62a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:37.534684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f1ab2-ab9c-4108-8877-51cb1918c62a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:37.541700 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f1ab2-ab9c-4108-8877-51cb1918c62a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:37.543140 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f1ab2-ab9c-4108-8877-51cb1918c62a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:37.557895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-783f1ab2-ab9c-4108-8877-51cb1918c62a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 18:50:37.557895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:37.557895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:37.557895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:37.557895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:37.557895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:37.557895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:37.557895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:37.557895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:37.557895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:37.557895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:37.557895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:37.557895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:50:37.557895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:50:37.557895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:50:37.557895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:50:37.558842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:37.558842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:37.558842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:50:37.558842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:37.558842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:50:37.558842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:50:37.558842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:37.558842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:37.558842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:37.558842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:37.558842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:37.558842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:37.558842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:37.558842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:37.558842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:37.558842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:37.558842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:37.558842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:37.560175 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:37.560175 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:37.560175 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:50:37.560175 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:50:37.560175 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:50:37.560175 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:50:37.560175 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:50:37.560175 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:50:37.560175 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 18:50:37.560175 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:37.560175 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 545/1099] 10.222.0.69 () {56 vars in 1093 bytes} [Thu Jul 14 18:50:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:37.565467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7daa3-0dc8-4302-8d09-75e38dec82b2 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:37.658091 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:37.663718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7daa3-0dc8-4302-8d09-75e38dec82b2 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:37.697988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5be3c-87e2-43fe-92c1-586bd30d55dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:37.698659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5be3c-87e2-43fe-92c1-586bd30d55dc None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:37.698945 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5be3c-87e2-43fe-92c1-586bd30d55dc None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:37.699256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5be3c-87e2-43fe-92c1-586bd30d55dc None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:37.700084 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5be3c-87e2-43fe-92c1-586bd30d55dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:37.700397 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5be3c-87e2-43fe-92c1-586bd30d55dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:37.701827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5be3c-87e2-43fe-92c1-586bd30d55dc None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:37.718806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 555/1100] 10.222.0.69 () {56 vars in 1133 bytes} [Thu Jul 14 18:50:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:37.724466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f1ab2-ab9c-4108-8877-51cb1918c62a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:37.822278 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f1ab2-ab9c-4108-8877-51cb1918c62a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:37.854782 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166bbd56-e45d-4085-93be-b0b6ee380dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:37.855448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166bbd56-e45d-4085-93be-b0b6ee380dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:37.855736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166bbd56-e45d-4085-93be-b0b6ee380dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:37.856014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166bbd56-e45d-4085-93be-b0b6ee380dd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:37.882264 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:37.921205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166bbd56-e45d-4085-93be-b0b6ee380dd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:37.922682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166bbd56-e45d-4085-93be-b0b6ee380dd2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:38.020965 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 546/1101] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:50:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:50:38.026695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5be3c-87e2-43fe-92c1-586bd30d55dc None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:38.122916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5be3c-87e2-43fe-92c1-586bd30d55dc None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:38.157341 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379e5f0-1263-4780-b2fc-31d73e34554d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:38.158157 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379e5f0-1263-4780-b2fc-31d73e34554d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:38.158438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379e5f0-1263-4780-b2fc-31d73e34554d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:38.158716 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379e5f0-1263-4780-b2fc-31d73e34554d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:38.168640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379e5f0-1263-4780-b2fc-31d73e34554d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:38.170108 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379e5f0-1263-4780-b2fc-31d73e34554d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:38.187676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-a379e5f0-1263-4780-b2fc-31d73e34554d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:38.187676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:38.187676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:38.187676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:38.187676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:38.187676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:38.187676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:38.187676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:38.187676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:38.187676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:38.187676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:38.187676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:38.187676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:50:38.187676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:50:38.187676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:50:38.188703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:50:38.188703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:38.188703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:38.188703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:38.188703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:38.188703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:38.188703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:38.188703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:38.188703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:38.188703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:38.188703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:38.188703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:38.188703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:38.188703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:38.188703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:38.188703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:38.188703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:38.190182 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:50:38.190182 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:50:38.190182 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:50:38.190182 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:38.191488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 556/1102] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:38.194845 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166bbd56-e45d-4085-93be-b0b6ee380dd2 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:38.252335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:38.291037 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166bbd56-e45d-4085-93be-b0b6ee380dd2 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:38.324224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac7457-c6fb-453a-ada5-0fc047aa4e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:38.324883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac7457-c6fb-453a-ada5-0fc047aa4e78 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:38.325171 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac7457-c6fb-453a-ada5-0fc047aa4e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:38.325451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac7457-c6fb-453a-ada5-0fc047aa4e78 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:38.326164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac7457-c6fb-453a-ada5-0fc047aa4e78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:38.326477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac7457-c6fb-453a-ada5-0fc047aa4e78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:38.327868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac7457-c6fb-453a-ada5-0fc047aa4e78 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:38.339412 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 547/1103] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:38.345002 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379e5f0-1263-4780-b2fc-31d73e34554d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:38.353862 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:38.443311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379e5f0-1263-4780-b2fc-31d73e34554d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:38.475610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112f165-ce85-4ab2-8374-ce739d14b009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:38.476408 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112f165-ce85-4ab2-8374-ce739d14b009 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:38.476692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112f165-ce85-4ab2-8374-ce739d14b009 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:38.476967 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112f165-ce85-4ab2-8374-ce739d14b009 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:38.483698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112f165-ce85-4ab2-8374-ce739d14b009 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:38.485151 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112f165-ce85-4ab2-8374-ce739d14b009 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:38.505836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-0112f165-ce85-4ab2-8374-ce739d14b009 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:38.505836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:38.505836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:38.505836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:38.505836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:38.505836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:38.505836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:38.505836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:38.505836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:38.505836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:38.505836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:38.505836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:38.505836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:50:38.505836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:50:38.505836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:50:38.506771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:50:38.506771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:38.506771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:38.506771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:38.506771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:38.506771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:38.506771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:38.506771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:38.506771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:38.506771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:38.506771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:38.506771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:38.506771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:38.506771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:38.506771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:38.506771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:38.506771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:38.507686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:50:38.507686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:50:38.507686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:50:38.507686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:50:38.507686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:50:38.507686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:50:38.507686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:50:38.507686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:50:38.507686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 557/1104] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:50:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:38.512949 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac7457-c6fb-453a-ada5-0fc047aa4e78 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:38.611824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac7457-c6fb-453a-ada5-0fc047aa4e78 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:38.631143 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:38.645409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f562d3-f451-45c1-9fd0-7f85ab0ee227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:38.647613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f562d3-f451-45c1-9fd0-7f85ab0ee227 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:38.647613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f562d3-f451-45c1-9fd0-7f85ab0ee227 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:38.647613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f562d3-f451-45c1-9fd0-7f85ab0ee227 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:38.647613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f562d3-f451-45c1-9fd0-7f85ab0ee227 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:38.647936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f562d3-f451-45c1-9fd0-7f85ab0ee227 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:38.649368 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f562d3-f451-45c1-9fd0-7f85ab0ee227 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:38.662157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 548/1105] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:50:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:38.668321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112f165-ce85-4ab2-8374-ce739d14b009 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:38.733307 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:38.766340 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112f165-ce85-4ab2-8374-ce739d14b009 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:38.800937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41692f-50a1-4fb1-b98a-996c5be074c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:38.801599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41692f-50a1-4fb1-b98a-996c5be074c8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:38.801889 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41692f-50a1-4fb1-b98a-996c5be074c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:38.802169 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41692f-50a1-4fb1-b98a-996c5be074c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:38.802999 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41692f-50a1-4fb1-b98a-996c5be074c8 None admin] RBAC: Adding query filter params (role.id=4def7095cc264443ab935a01966235a0, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=8d27d6df5cb64697adebd961dfbec324) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:38.803349 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41692f-50a1-4fb1-b98a-996c5be074c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:38.804787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41692f-50a1-4fb1-b98a-996c5be074c8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:38.814474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 558/1106] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:50:38 2022] GET /identity/v3/role_assignments?user.id=8d27d6df5cb64697adebd961dfbec324&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=4def7095cc264443ab935a01966235a0 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:38.960002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3cf76-f01b-4c3d-ba28-9eec7f9f52e7 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:38.960534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3cf76-f01b-4c3d-ba28-9eec7f9f52e7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:38.960638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3cf76-f01b-4c3d-ba28-9eec7f9f52e7 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:38.961983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 549/1107] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:38.967307 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0d8bd-41a3-4c44-a482-ed56f49ea39b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:38.967605 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0d8bd-41a3-4c44-a482-ed56f49ea39b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:38.967889 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0d8bd-41a3-4c44-a482-ed56f49ea39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:38.985455 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-29c0d8bd-41a3-4c44-a482-ed56f49ea39b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:38.986566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-29c0d8bd-41a3-4c44-a482-ed56f49ea39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:39.055615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 559/1108] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:39.080344 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3cf76-f01b-4c3d-ba28-9eec7f9f52e7 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:39.184673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3cf76-f01b-4c3d-ba28-9eec7f9f52e7 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:39.220122 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c3fe4-8e7a-465d-add9-8a6d74e8a171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:39.222119 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c3fe4-8e7a-465d-add9-8a6d74e8a171 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:39.222119 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c3fe4-8e7a-465d-add9-8a6d74e8a171 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:39.222119 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c3fe4-8e7a-465d-add9-8a6d74e8a171 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:39.223356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c3fe4-8e7a-465d-add9-8a6d74e8a171 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:39.224802 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c3fe4-8e7a-465d-add9-8a6d74e8a171 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:39.226491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 550/1109] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:39.233280 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0d8bd-41a3-4c44-a482-ed56f49ea39b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:39.294458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:39.336624 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0d8bd-41a3-4c44-a482-ed56f49ea39b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:39.370212 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06b448-3c3e-448e-8af8-5944772e4a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:39.370896 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06b448-3c3e-448e-8af8-5944772e4a64 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:39.371195 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06b448-3c3e-448e-8af8-5944772e4a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:39.371499 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06b448-3c3e-448e-8af8-5944772e4a64 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:39.372210 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06b448-3c3e-448e-8af8-5944772e4a64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:39.373636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06b448-3c3e-448e-8af8-5944772e4a64 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:39.466763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 560/1110] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 14 18:50:39 2022] POST /identity/v3/users => generated 286 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:50:41.527847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93f103-c150-4e4b-973b-e19f77f51572 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:41.528382 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93f103-c150-4e4b-973b-e19f77f51572 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:41.528382 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93f103-c150-4e4b-973b-e19f77f51572 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:41.529756 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 551/1111] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:41.535397 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e28c0-9efa-46c1-a23e-ab7b3be3629e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:41.535773 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e28c0-9efa-46c1-a23e-ab7b3be3629e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:41.535972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e28c0-9efa-46c1-a23e-ab7b3be3629e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:41.552953 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-8f5e28c0-9efa-46c1-a23e-ab7b3be3629e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:41.554149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8f5e28c0-9efa-46c1-a23e-ab7b3be3629e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:41.618218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 561/1112] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:50:41.640820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93f103-c150-4e4b-973b-e19f77f51572 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:41.690672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:41.740786 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93f103-c150-4e4b-973b-e19f77f51572 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:41.772700 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48611e-2b16-4373-a9c4-1e988bdb639e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:41.773552 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48611e-2b16-4373-a9c4-1e988bdb639e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:41.773843 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48611e-2b16-4373-a9c4-1e988bdb639e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:41.774123 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48611e-2b16-4373-a9c4-1e988bdb639e None admin] PATH_INFO: `/v3/roles/394e165201dc41c8b3255de480821b1f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:41.775629 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48611e-2b16-4373-a9c4-1e988bdb639e None admin] RBAC: Authorizing `identity:get_role(role_id=394e165201dc41c8b3255de480821b1f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:41.777267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48611e-2b16-4373-a9c4-1e988bdb639e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:41.778671 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 552/1113] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:50:41 2022] GET /identity/v3/roles/394e165201dc41c8b3255de480821b1f => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:41.784917 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e28c0-9efa-46c1-a23e-ab7b3be3629e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:41.831406 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:41.882376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e28c0-9efa-46c1-a23e-ab7b3be3629e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:41.915709 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffeeb95-bc44-4a55-b51b-69a731925ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:41.916574 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffeeb95-bc44-4a55-b51b-69a731925ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:41.916864 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffeeb95-bc44-4a55-b51b-69a731925ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:41.917144 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffeeb95-bc44-4a55-b51b-69a731925ab2 None admin] PATH_INFO: `/v3/users/fecd049357054d5cae4f136aa74ba6d6` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:41.930393 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffeeb95-bc44-4a55-b51b-69a731925ab2 None admin] RBAC: Authorizing `identity:get_user(user_id=fecd049357054d5cae4f136aa74ba6d6)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:41.931874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffeeb95-bc44-4a55-b51b-69a731925ab2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:41.933536 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 562/1114] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:50:41 2022] GET /identity/v3/users/fecd049357054d5cae4f136aa74ba6d6 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:41.939419 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48611e-2b16-4373-a9c4-1e988bdb639e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:42.037805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48611e-2b16-4373-a9c4-1e988bdb639e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:42.053217 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:42.070098 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf8595-7c0c-4cc0-920a-78cd78d13ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:42.070771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf8595-7c0c-4cc0-920a-78cd78d13ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:42.071034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf8595-7c0c-4cc0-920a-78cd78d13ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:42.071344 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf8595-7c0c-4cc0-920a-78cd78d13ac0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:42.137232 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf8595-7c0c-4cc0-920a-78cd78d13ac0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:42.138698 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf8595-7c0c-4cc0-920a-78cd78d13ac0 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:42.232067 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 553/1115] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:50:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:50:42.238455 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffeeb95-bc44-4a55-b51b-69a731925ab2 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:42.334137 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffeeb95-bc44-4a55-b51b-69a731925ab2 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:42.336429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:42.367697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecdf45b-b070-426e-bb20-b0ccd37f4089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:42.368543 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdf45b-b070-426e-bb20-b0ccd37f4089 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:42.368833 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdf45b-b070-426e-bb20-b0ccd37f4089 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:42.369114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdf45b-b070-426e-bb20-b0ccd37f4089 None admin] PATH_INFO: `/v3/projects/25b5df177a964556af065fb2b1f1ca46` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:42.370290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecdf45b-b070-426e-bb20-b0ccd37f4089 None admin] RBAC: Authorizing `identity:get_project(project_id=25b5df177a964556af065fb2b1f1ca46)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:42.371770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecdf45b-b070-426e-bb20-b0ccd37f4089 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:42.373828 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 563/1116] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:50:42 2022] GET /identity/v3/projects/25b5df177a964556af065fb2b1f1ca46 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:42.379397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf8595-7c0c-4cc0-920a-78cd78d13ac0 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:42.396044 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:42.474040 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf8595-7c0c-4cc0-920a-78cd78d13ac0 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:42.506823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836e66a-c064-45b3-8ad6-2222ff4ab1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:42.507491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836e66a-c064-45b3-8ad6-2222ff4ab1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:42.507778 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836e66a-c064-45b3-8ad6-2222ff4ab1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:42.508053 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836e66a-c064-45b3-8ad6-2222ff4ab1f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:42.508868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836e66a-c064-45b3-8ad6-2222ff4ab1f5 None admin] RBAC: Adding query filter params (role.id=394e165201dc41c8b3255de480821b1f, scope.project.id=25b5df177a964556af065fb2b1f1ca46, user.id=fecd049357054d5cae4f136aa74ba6d6) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:42.509177 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836e66a-c064-45b3-8ad6-2222ff4ab1f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:42.510595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836e66a-c064-45b3-8ad6-2222ff4ab1f5 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:42.515855 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 554/1117] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:50:42 2022] GET /identity/v3/role_assignments?user.id=fecd049357054d5cae4f136aa74ba6d6&scope.project.id=25b5df177a964556af065fb2b1f1ca46&role.id=394e165201dc41c8b3255de480821b1f => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:44.488950 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc3809-e32d-43f2-8d4b-c5e647ff6929 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:44.489475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc3809-e32d-43f2-8d4b-c5e647ff6929 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:44.489566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc3809-e32d-43f2-8d4b-c5e647ff6929 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:44.490901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 564/1118] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:44.495987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a81a9-883e-4f2b-b1f6-37c8439cc29d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:44.496411 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a81a9-883e-4f2b-b1f6-37c8439cc29d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:44.496552 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a81a9-883e-4f2b-b1f6-37c8439cc29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:44.513351 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ba3a81a9-883e-4f2b-b1f6-37c8439cc29d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:44.514516 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ba3a81a9-883e-4f2b-b1f6-37c8439cc29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:44.578025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 555/1119] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:44.599990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc3809-e32d-43f2-8d4b-c5e647ff6929 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:44.698485 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc3809-e32d-43f2-8d4b-c5e647ff6929 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:44.731142 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9325082-00d1-495b-bdc4-39c0d7226ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:44.732026 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9325082-00d1-495b-bdc4-39c0d7226ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:44.732314 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9325082-00d1-495b-bdc4-39c0d7226ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:44.732598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9325082-00d1-495b-bdc4-39c0d7226ff0 None admin] PATH_INFO: `/v3/roles/394e165201dc41c8b3255de480821b1f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:44.734077 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9325082-00d1-495b-bdc4-39c0d7226ff0 None admin] RBAC: Authorizing `identity:get_role(role_id=394e165201dc41c8b3255de480821b1f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:44.735480 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9325082-00d1-495b-bdc4-39c0d7226ff0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:44.737480 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 565/1120] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:50:44 2022] GET /identity/v3/roles/394e165201dc41c8b3255de480821b1f => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:44.743113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a81a9-883e-4f2b-b1f6-37c8439cc29d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:44.754328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:44.838384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a81a9-883e-4f2b-b1f6-37c8439cc29d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:44.872338 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47797626-5057-4e33-ad15-663dab86a232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:44.873199 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47797626-5057-4e33-ad15-663dab86a232 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:44.873483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47797626-5057-4e33-ad15-663dab86a232 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:44.873761 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47797626-5057-4e33-ad15-663dab86a232 None admin] PATH_INFO: `/v3/users/fecd049357054d5cae4f136aa74ba6d6` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:44.874919 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47797626-5057-4e33-ad15-663dab86a232 None admin] RBAC: Authorizing `identity:get_user(user_id=fecd049357054d5cae4f136aa74ba6d6)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:44.876408 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47797626-5057-4e33-ad15-663dab86a232 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:44.878007 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 556/1121] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:50:44 2022] GET /identity/v3/users/fecd049357054d5cae4f136aa74ba6d6 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:44.883914 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9325082-00d1-495b-bdc4-39c0d7226ff0 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:44.927747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:44.982315 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9325082-00d1-495b-bdc4-39c0d7226ff0 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:45.015255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed0c76-166e-4f55-a0c6-2da30b258a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:45.016117 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed0c76-166e-4f55-a0c6-2da30b258a2a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:45.016401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed0c76-166e-4f55-a0c6-2da30b258a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:45.016677 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed0c76-166e-4f55-a0c6-2da30b258a2a None admin] PATH_INFO: `/v3/projects/25b5df177a964556af065fb2b1f1ca46` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:45.018371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed0c76-166e-4f55-a0c6-2da30b258a2a None admin] RBAC: Authorizing `identity:get_project(project_id=25b5df177a964556af065fb2b1f1ca46)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:45.019845 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed0c76-166e-4f55-a0c6-2da30b258a2a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:45.021533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 566/1122] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:50:44 2022] GET /identity/v3/projects/25b5df177a964556af065fb2b1f1ca46 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:45.027624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47797626-5057-4e33-ad15-663dab86a232 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:45.102280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:45.122831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47797626-5057-4e33-ad15-663dab86a232 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:45.156284 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46e5fc-28b2-40ef-b1de-6a3935bcee5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:45.157163 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46e5fc-28b2-40ef-b1de-6a3935bcee5d None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:45.157467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46e5fc-28b2-40ef-b1de-6a3935bcee5d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:45.157743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46e5fc-28b2-40ef-b1de-6a3935bcee5d None admin] PATH_INFO: `/v3/projects/25b5df177a964556af065fb2b1f1ca46/users/fecd049357054d5cae4f136aa74ba6d6/roles/394e165201dc41c8b3255de480821b1f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:45.159908 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46e5fc-28b2-40ef-b1de-6a3935bcee5d None admin] RBAC: Authorizing `identity:create_grant(project_id=25b5df177a964556af065fb2b1f1ca46, user_id=fecd049357054d5cae4f136aa74ba6d6, role_id=394e165201dc41c8b3255de480821b1f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:45.161316 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46e5fc-28b2-40ef-b1de-6a3935bcee5d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:45.259999 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 557/1123] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:50:45 2022] PUT /identity/v3/projects/25b5df177a964556af065fb2b1f1ca46/users/fecd049357054d5cae4f136aa74ba6d6/roles/394e165201dc41c8b3255de480821b1f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:50:47.207670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6a605-d2e3-47d3-b020-9fbaa63a61bb None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:47.208179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6a605-d2e3-47d3-b020-9fbaa63a61bb None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:47.208288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6a605-d2e3-47d3-b020-9fbaa63a61bb None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:47.209592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 567/1124] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:47.214616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23210484-c62d-4f19-8e68-b8f06e4117bf None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:47.215004 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23210484-c62d-4f19-8e68-b8f06e4117bf None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:47.215217 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23210484-c62d-4f19-8e68-b8f06e4117bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:47.232330 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-23210484-c62d-4f19-8e68-b8f06e4117bf None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:47.233522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-23210484-c62d-4f19-8e68-b8f06e4117bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:47.298850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 558/1125] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:47.322310 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6a605-d2e3-47d3-b020-9fbaa63a61bb None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:47.419086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6a605-d2e3-47d3-b020-9fbaa63a61bb None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:47.428460 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:47.452316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dacc33-5653-4686-98e4-e58ceeafdd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:47.453154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dacc33-5653-4686-98e4-e58ceeafdd33 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:47.453482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dacc33-5653-4686-98e4-e58ceeafdd33 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:47.453758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dacc33-5653-4686-98e4-e58ceeafdd33 None admin] PATH_INFO: `/v3/roles/394e165201dc41c8b3255de480821b1f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:47.455219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dacc33-5653-4686-98e4-e58ceeafdd33 None admin] RBAC: Authorizing `identity:get_role(role_id=394e165201dc41c8b3255de480821b1f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:47.456586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dacc33-5653-4686-98e4-e58ceeafdd33 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:47.457930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 568/1126] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:50:47 2022] GET /identity/v3/roles/394e165201dc41c8b3255de480821b1f => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:47.464001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23210484-c62d-4f19-8e68-b8f06e4117bf None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:47.526849 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:47.559916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23210484-c62d-4f19-8e68-b8f06e4117bf None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:47.603311 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0739e1-d42b-4bf5-b519-0e16b43f04f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:47.604179 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0739e1-d42b-4bf5-b519-0e16b43f04f6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:47.604463 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0739e1-d42b-4bf5-b519-0e16b43f04f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:47.604737 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0739e1-d42b-4bf5-b519-0e16b43f04f6 None admin] PATH_INFO: `/v3/users/fecd049357054d5cae4f136aa74ba6d6` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:47.605907 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0739e1-d42b-4bf5-b519-0e16b43f04f6 None admin] RBAC: Authorizing `identity:get_user(user_id=fecd049357054d5cae4f136aa74ba6d6)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:47.607362 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0739e1-d42b-4bf5-b519-0e16b43f04f6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:47.609003 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 559/1127] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:50:47 2022] GET /identity/v3/users/fecd049357054d5cae4f136aa74ba6d6 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:47.614997 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dacc33-5653-4686-98e4-e58ceeafdd33 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:47.711999 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dacc33-5653-4686-98e4-e58ceeafdd33 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:47.744306 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2be5a9-1581-4b5f-8ba8-6e40fa99ed03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:47.744956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2be5a9-1581-4b5f-8ba8-6e40fa99ed03 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:47.745241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2be5a9-1581-4b5f-8ba8-6e40fa99ed03 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:47.745520 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2be5a9-1581-4b5f-8ba8-6e40fa99ed03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:47.789772 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:47.811878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2be5a9-1581-4b5f-8ba8-6e40fa99ed03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:47.813286 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2be5a9-1581-4b5f-8ba8-6e40fa99ed03 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:47.919010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 569/1128] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:50:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:50:47.925729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0739e1-d42b-4bf5-b519-0e16b43f04f6 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:48.024361 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0739e1-d42b-4bf5-b519-0e16b43f04f6 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:48.057947 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405d12c-8a5c-4c93-8ce7-985084ea0705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:48.058804 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405d12c-8a5c-4c93-8ce7-985084ea0705 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:48.059091 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405d12c-8a5c-4c93-8ce7-985084ea0705 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:48.059402 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405d12c-8a5c-4c93-8ce7-985084ea0705 None admin] PATH_INFO: `/v3/projects/25b5df177a964556af065fb2b1f1ca46` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:48.060581 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405d12c-8a5c-4c93-8ce7-985084ea0705 None admin] RBAC: Authorizing `identity:get_project(project_id=25b5df177a964556af065fb2b1f1ca46)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:48.062061 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405d12c-8a5c-4c93-8ce7-985084ea0705 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:48.063818 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 560/1129] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:50:47 2022] GET /identity/v3/projects/25b5df177a964556af065fb2b1f1ca46 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:48.069855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2be5a9-1581-4b5f-8ba8-6e40fa99ed03 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:48.147158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:48.170459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2be5a9-1581-4b5f-8ba8-6e40fa99ed03 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:48.202781 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f2eb4-41ee-4c0a-9ce7-cc3928a98cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:48.203456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f2eb4-41ee-4c0a-9ce7-cc3928a98cfc None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:48.203741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f2eb4-41ee-4c0a-9ce7-cc3928a98cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:48.204016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f2eb4-41ee-4c0a-9ce7-cc3928a98cfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:48.205327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f2eb4-41ee-4c0a-9ce7-cc3928a98cfc None admin] RBAC: Adding query filter params (role.id=394e165201dc41c8b3255de480821b1f, scope.project.id=25b5df177a964556af065fb2b1f1ca46, user.id=fecd049357054d5cae4f136aa74ba6d6) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:48.205639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f2eb4-41ee-4c0a-9ce7-cc3928a98cfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:48.207077 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f2eb4-41ee-4c0a-9ce7-cc3928a98cfc None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:48.212892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 570/1130] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:50:48 2022] GET /identity/v3/role_assignments?user.id=fecd049357054d5cae4f136aa74ba6d6&scope.project.id=25b5df177a964556af065fb2b1f1ca46&role.id=394e165201dc41c8b3255de480821b1f => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:50.248551 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d47632-a9ec-4e81-b9d4-a022d3e36486 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:50.249084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d47632-a9ec-4e81-b9d4-a022d3e36486 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:50.249084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d47632-a9ec-4e81-b9d4-a022d3e36486 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:50.250453 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 561/1131] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:50.256254 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b19de9-9fe8-4c8d-80af-c5d8cd892664 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:50.256664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b19de9-9fe8-4c8d-80af-c5d8cd892664 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:50.256762 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b19de9-9fe8-4c8d-80af-c5d8cd892664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:50.272964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-a4b19de9-9fe8-4c8d-80af-c5d8cd892664 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:50.274131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a4b19de9-9fe8-4c8d-80af-c5d8cd892664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:50.337622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 571/1132] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:50.359943 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d47632-a9ec-4e81-b9d4-a022d3e36486 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:50.364206 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:50.458681 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d47632-a9ec-4e81-b9d4-a022d3e36486 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:50.492215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744f791-9ea1-41c3-b89c-ff27abbe59be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:50.492987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744f791-9ea1-41c3-b89c-ff27abbe59be None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:50.493295 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744f791-9ea1-41c3-b89c-ff27abbe59be None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:50.493572 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744f791-9ea1-41c3-b89c-ff27abbe59be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:50.494725 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744f791-9ea1-41c3-b89c-ff27abbe59be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:50.496173 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744f791-9ea1-41c3-b89c-ff27abbe59be None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:50.497806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 562/1133] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:50.503789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b19de9-9fe8-4c8d-80af-c5d8cd892664 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:50.571433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:50.600875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b19de9-9fe8-4c8d-80af-c5d8cd892664 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:50.633537 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829ea67-95d2-41b9-b662-fdb866e65d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:50.634196 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829ea67-95d2-41b9-b662-fdb866e65d0c None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:50.634484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829ea67-95d2-41b9-b662-fdb866e65d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:50.634766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829ea67-95d2-41b9-b662-fdb866e65d0c None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:50.635508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829ea67-95d2-41b9-b662-fdb866e65d0c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:50.636943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829ea67-95d2-41b9-b662-fdb866e65d0c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:50.712638 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 572/1134] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 14 18:50:50 2022] POST /identity/v3/projects => generated 311 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:50:52.732030 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780eabe1-b6ed-4461-be44-858c34e4ab12 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:52.732583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780eabe1-b6ed-4461-be44-858c34e4ab12 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:52.732583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780eabe1-b6ed-4461-be44-858c34e4ab12 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:52.733939 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 563/1135] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:52.739854 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6631741-cb58-49a2-81cd-7a7b739c426c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:52.740278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6631741-cb58-49a2-81cd-7a7b739c426c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:52.740422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6631741-cb58-49a2-81cd-7a7b739c426c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:52.757391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-a6631741-cb58-49a2-81cd-7a7b739c426c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:52.758574 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a6631741-cb58-49a2-81cd-7a7b739c426c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:52.822976 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 573/1136] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:52.846981 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780eabe1-b6ed-4461-be44-858c34e4ab12 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:52.918486 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:52.949701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780eabe1-b6ed-4461-be44-858c34e4ab12 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:52.982308 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a8727-19ff-404d-99d1-dbad3c47adac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:52.983024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a8727-19ff-404d-99d1-dbad3c47adac None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:52.983338 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a8727-19ff-404d-99d1-dbad3c47adac None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:52.983613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a8727-19ff-404d-99d1-dbad3c47adac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:52.984755 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a8727-19ff-404d-99d1-dbad3c47adac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:52.987027 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a8727-19ff-404d-99d1-dbad3c47adac None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:52.988741 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 564/1137] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:50:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:52.994584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6631741-cb58-49a2-81cd-7a7b739c426c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:53.049967 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:53.094459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6631741-cb58-49a2-81cd-7a7b739c426c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:53.128098 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e2058-0740-48b9-8753-6086e63ddf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:53.128757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e2058-0740-48b9-8753-6086e63ddf41 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:53.129040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e2058-0740-48b9-8753-6086e63ddf41 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:53.129316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e2058-0740-48b9-8753-6086e63ddf41 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:53.130006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e2058-0740-48b9-8753-6086e63ddf41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:53.131463 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e2058-0740-48b9-8753-6086e63ddf41 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:53.226254 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 574/1138] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 14 18:50:52 2022] POST /identity/v3/users => generated 286 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:50:55.246564 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc348e-1b03-400e-b560-ebb49229c584 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:55.247109 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc348e-1b03-400e-b560-ebb49229c584 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:55.247109 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc348e-1b03-400e-b560-ebb49229c584 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:55.248547 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 565/1139] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:55.254201 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5499d1a-d0be-47a8-8363-ecf164cd5b9d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:55.254620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5499d1a-d0be-47a8-8363-ecf164cd5b9d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:55.254721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5499d1a-d0be-47a8-8363-ecf164cd5b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:55.271730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c5499d1a-d0be-47a8-8363-ecf164cd5b9d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:55.272899 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c5499d1a-d0be-47a8-8363-ecf164cd5b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:55.354407 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 575/1140] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:55.378130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc348e-1b03-400e-b560-ebb49229c584 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:55.482220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc348e-1b03-400e-b560-ebb49229c584 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:55.515188 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad24b19-6513-4ee3-b234-5f0410f2ed62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:55.515910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad24b19-6513-4ee3-b234-5f0410f2ed62 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:55.516196 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad24b19-6513-4ee3-b234-5f0410f2ed62 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:55.516489 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad24b19-6513-4ee3-b234-5f0410f2ed62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:55.523729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:55.526596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad24b19-6513-4ee3-b234-5f0410f2ed62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:50:55.534548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-cad24b19-6513-4ee3-b234-5f0410f2ed62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:50:55.534548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:55.534548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:55.534548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:55.534548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:55.534548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:55.534548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:55.534548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:55.534548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:55.534548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:55.534548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:55.534548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:55.534548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:50:55.534548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:50:55.534548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:50:55.534548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:50:55.537157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:50:55.537157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:50:55.537157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:50:55.537157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:50:55.537157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:55.537157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:55.537157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:55.537157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:55.537157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:55.537157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:55.537157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:55.537157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:55.537157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:55.537157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:55.537157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:55.537157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:55.538269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:55.538269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:55.538269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:55.538269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:55.538269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:50:55.538269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:50:55.538269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:50:55.538269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:50:55.538269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:50:55.538269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:50:55.538269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:50:55.538269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:55.538269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 566/1141] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:50:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:55.541928 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5499d1a-d0be-47a8-8363-ecf164cd5b9d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:55.581321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:55.638649 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5499d1a-d0be-47a8-8363-ecf164cd5b9d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:55.670916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bac75e-4250-4e7a-8e05-ca88797c204f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:55.671603 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bac75e-4250-4e7a-8e05-ca88797c204f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:55.671889 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bac75e-4250-4e7a-8e05-ca88797c204f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:55.672166 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bac75e-4250-4e7a-8e05-ca88797c204f None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:55.673516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bac75e-4250-4e7a-8e05-ca88797c204f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:55.673827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bac75e-4250-4e7a-8e05-ca88797c204f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:55.675301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bac75e-4250-4e7a-8e05-ca88797c204f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:55.683049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 576/1142] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:50:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:55.689381 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad24b19-6513-4ee3-b234-5f0410f2ed62 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:55.786552 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad24b19-6513-4ee3-b234-5f0410f2ed62 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:55.818560 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f29554-6cb7-41ab-ab13-77bedf52623f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:55.819433 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f29554-6cb7-41ab-ab13-77bedf52623f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:55.819745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f29554-6cb7-41ab-ab13-77bedf52623f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:55.820022 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f29554-6cb7-41ab-ab13-77bedf52623f None admin] PATH_INFO: `/v3/users/27e2cbd4b28543339e14c4783d92120c` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:55.833102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f29554-6cb7-41ab-ab13-77bedf52623f None admin] RBAC: Authorizing `identity:get_user(user_id=27e2cbd4b28543339e14c4783d92120c)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:55.834530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f29554-6cb7-41ab-ab13-77bedf52623f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:55.836169 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 567/1143] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:50:55 2022] GET /identity/v3/users/27e2cbd4b28543339e14c4783d92120c => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:55.842020 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bac75e-4250-4e7a-8e05-ca88797c204f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:55.936004 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:55.939443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bac75e-4250-4e7a-8e05-ca88797c204f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:55.974838 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf66e00-81e1-4fbd-9dd6-77ef92baefee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:55.975500 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf66e00-81e1-4fbd-9dd6-77ef92baefee None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:55.975787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf66e00-81e1-4fbd-9dd6-77ef92baefee None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:55.976064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf66e00-81e1-4fbd-9dd6-77ef92baefee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:56.041162 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf66e00-81e1-4fbd-9dd6-77ef92baefee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:56.042554 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf66e00-81e1-4fbd-9dd6-77ef92baefee None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:56.132185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:56.135710 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 577/1144] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:50:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:50:56.142369 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f29554-6cb7-41ab-ab13-77bedf52623f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:56.199870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:56.241020 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f29554-6cb7-41ab-ab13-77bedf52623f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:56.273460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f975b-c0bc-4eb9-a58e-d9ee8a7e5d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:56.274318 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f975b-c0bc-4eb9-a58e-d9ee8a7e5d1c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:56.274605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f975b-c0bc-4eb9-a58e-d9ee8a7e5d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:56.274883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f975b-c0bc-4eb9-a58e-d9ee8a7e5d1c None admin] PATH_INFO: `/v3/projects/f4d2a32ba5f147c5846f28319bfa541b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:56.276068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f975b-c0bc-4eb9-a58e-d9ee8a7e5d1c None admin] RBAC: Authorizing `identity:get_project(project_id=f4d2a32ba5f147c5846f28319bfa541b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:56.277767 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f975b-c0bc-4eb9-a58e-d9ee8a7e5d1c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:56.279477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 568/1145] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:50:56 2022] GET /identity/v3/projects/f4d2a32ba5f147c5846f28319bfa541b => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:56.285475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf66e00-81e1-4fbd-9dd6-77ef92baefee None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:56.379668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf66e00-81e1-4fbd-9dd6-77ef92baefee None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:56.411934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c50b2-b120-42c6-8d0f-1ba61ec42658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:56.412580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c50b2-b120-42c6-8d0f-1ba61ec42658 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:56.412864 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c50b2-b120-42c6-8d0f-1ba61ec42658 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:56.413139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c50b2-b120-42c6-8d0f-1ba61ec42658 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:56.413952 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c50b2-b120-42c6-8d0f-1ba61ec42658 None admin] RBAC: Adding query filter params (role.id=27fc14631c494ca0b2dc7f708be016c2, scope.project.id=f4d2a32ba5f147c5846f28319bfa541b, user.id=27e2cbd4b28543339e14c4783d92120c) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:56.414260 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c50b2-b120-42c6-8d0f-1ba61ec42658 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:56.415727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c50b2-b120-42c6-8d0f-1ba61ec42658 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:56.420936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 578/1146] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:50:56 2022] GET /identity/v3/role_assignments?user.id=27e2cbd4b28543339e14c4783d92120c&scope.project.id=f4d2a32ba5f147c5846f28319bfa541b&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:58.372880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2baf16-df96-440d-8867-ab427711b71d None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:58.373413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2baf16-df96-440d-8867-ab427711b71d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:58.373514 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2baf16-df96-440d-8867-ab427711b71d None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:58.374839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 569/1147] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:50:58.380506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85e69f-7bde-4df4-98bc-ab3141b02f4f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:58.380885 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85e69f-7bde-4df4-98bc-ab3141b02f4f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:58.381059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85e69f-7bde-4df4-98bc-ab3141b02f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:58.397862 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-bf85e69f-7bde-4df4-98bc-ab3141b02f4f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:50:58.399039 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bf85e69f-7bde-4df4-98bc-ab3141b02f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:50:58.455491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:58.463444 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 579/1148] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:50:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:50:58.484745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2baf16-df96-440d-8867-ab427711b71d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:58.585863 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2baf16-df96-440d-8867-ab427711b71d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:58.610101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:58.618693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac36989-4592-427d-9911-86da3d27af04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:58.619508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac36989-4592-427d-9911-86da3d27af04 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:58.619794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac36989-4592-427d-9911-86da3d27af04 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:58.620068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac36989-4592-427d-9911-86da3d27af04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:58.630355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac36989-4592-427d-9911-86da3d27af04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:50:58.638174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-bac36989-4592-427d-9911-86da3d27af04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:50:58.638174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:50:58.638174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:50:58.638174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:50:58.638174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:50:58.638174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:50:58.638174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:50:58.638174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:50:58.638174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:50:58.638174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:50:58.638174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:50:58.638174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:50:58.638174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:50:58.638174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:50:58.638174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:50:58.638174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:50:58.639328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:50:58.639328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:50:58.639328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:50:58.639328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:50:58.639328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:50:58.639328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:50:58.639328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:50:58.639328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:50:58.639328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:50:58.639328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:50:58.639328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:50:58.639328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:50:58.639328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:50:58.639328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:50:58.639328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:50:58.639328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:50:58.640192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:50:58.640192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:50:58.640192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:50:58.640192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:50:58.640192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:50:58.640192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:50:58.640192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:50:58.640192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:50:58.640192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:50:58.640192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:50:58.640192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:50:58.640192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:50:58.640192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 570/1149] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:50:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:50:58.645520 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85e69f-7bde-4df4-98bc-ab3141b02f4f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:58.739827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85e69f-7bde-4df4-98bc-ab3141b02f4f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:58.772324 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cf1d1-1381-42af-aec9-2b2c3ef07604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:58.772977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cf1d1-1381-42af-aec9-2b2c3ef07604 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:58.773261 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cf1d1-1381-42af-aec9-2b2c3ef07604 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:58.773537 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cf1d1-1381-42af-aec9-2b2c3ef07604 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:58.774268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87cf1d1-1381-42af-aec9-2b2c3ef07604 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:50:58.774573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87cf1d1-1381-42af-aec9-2b2c3ef07604 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:58.775975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87cf1d1-1381-42af-aec9-2b2c3ef07604 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:58.784465 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 580/1150] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:50:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:58.790171 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac36989-4592-427d-9911-86da3d27af04 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:58.886962 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac36989-4592-427d-9911-86da3d27af04 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:58.919106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065af665-aba5-428f-9ce5-8470d624efea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:58.919990 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065af665-aba5-428f-9ce5-8470d624efea None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:58.920278 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065af665-aba5-428f-9ce5-8470d624efea None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:58.920554 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065af665-aba5-428f-9ce5-8470d624efea None admin] PATH_INFO: `/v3/users/27e2cbd4b28543339e14c4783d92120c` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:58.921732 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065af665-aba5-428f-9ce5-8470d624efea None admin] RBAC: Authorizing `identity:get_user(user_id=27e2cbd4b28543339e14c4783d92120c)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:58.923752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065af665-aba5-428f-9ce5-8470d624efea None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:58.925344 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 571/1151] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:50:58 2022] GET /identity/v3/users/27e2cbd4b28543339e14c4783d92120c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:58.931025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cf1d1-1381-42af-aec9-2b2c3ef07604 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:58.985790 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:59.031751 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cf1d1-1381-42af-aec9-2b2c3ef07604 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:59.064815 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a4551-86a9-4b74-a3d0-0f2f10679500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:59.065663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a4551-86a9-4b74-a3d0-0f2f10679500 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:59.065943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a4551-86a9-4b74-a3d0-0f2f10679500 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:59.066212 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a4551-86a9-4b74-a3d0-0f2f10679500 None admin] PATH_INFO: `/v3/projects/f4d2a32ba5f147c5846f28319bfa541b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:59.067415 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a4551-86a9-4b74-a3d0-0f2f10679500 None admin] RBAC: Authorizing `identity:get_project(project_id=f4d2a32ba5f147c5846f28319bfa541b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:59.068876 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a4551-86a9-4b74-a3d0-0f2f10679500 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:59.070562 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 581/1152] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:50:58 2022] GET /identity/v3/projects/f4d2a32ba5f147c5846f28319bfa541b => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:50:59.076512 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065af665-aba5-428f-9ce5-8470d624efea None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:50:59.130725 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:50:59.175064 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065af665-aba5-428f-9ce5-8470d624efea None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:50:59.207483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b41176-f4b8-4f8c-a3a0-31c92d1bcd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:50:59.208368 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b41176-f4b8-4f8c-a3a0-31c92d1bcd9a None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:50:59.208656 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b41176-f4b8-4f8c-a3a0-31c92d1bcd9a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:50:59.208940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b41176-f4b8-4f8c-a3a0-31c92d1bcd9a None admin] PATH_INFO: `/v3/projects/f4d2a32ba5f147c5846f28319bfa541b/users/27e2cbd4b28543339e14c4783d92120c/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:50:59.210884 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b41176-f4b8-4f8c-a3a0-31c92d1bcd9a None admin] RBAC: Authorizing `identity:create_grant(project_id=f4d2a32ba5f147c5846f28319bfa541b, user_id=27e2cbd4b28543339e14c4783d92120c, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:50:59.212345 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b41176-f4b8-4f8c-a3a0-31c92d1bcd9a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:50:59.283737 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 572/1153] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:50:59 2022] PUT /identity/v3/projects/f4d2a32ba5f147c5846f28319bfa541b/users/27e2cbd4b28543339e14c4783d92120c/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:51:01.272143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d159c-727c-48b4-829b-540cbe43bdec None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:01.272634 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d159c-727c-48b4-829b-540cbe43bdec None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:01.272634 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d159c-727c-48b4-829b-540cbe43bdec None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:01.274017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 582/1154] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:51:01.279724 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbeb84d-9dc9-4e04-9a0d-4b11025c8f19 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:01.280157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbeb84d-9dc9-4e04-9a0d-4b11025c8f19 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:01.280295 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbeb84d-9dc9-4e04-9a0d-4b11025c8f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:01.297008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-2fbeb84d-9dc9-4e04-9a0d-4b11025c8f19 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:51:01.298789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2fbeb84d-9dc9-4e04-9a0d-4b11025c8f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:51:01.362358 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 573/1155] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:51:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:51:01.385082 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d159c-727c-48b4-829b-540cbe43bdec None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:01.491736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d159c-727c-48b4-829b-540cbe43bdec None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:01.513887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:01.524664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78b9a5-a469-42e1-a2f1-4c67ad7634ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:01.525394 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78b9a5-a469-42e1-a2f1-4c67ad7634ad None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:01.525678 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78b9a5-a469-42e1-a2f1-4c67ad7634ad None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:01.525960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78b9a5-a469-42e1-a2f1-4c67ad7634ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:01.536477 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78b9a5-a469-42e1-a2f1-4c67ad7634ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:51:01.544064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-ed78b9a5-a469-42e1-a2f1-4c67ad7634ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:51:01.544064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:51:01.544064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:51:01.544064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:51:01.544064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:51:01.544064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:51:01.544064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:51:01.544064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:51:01.544064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:51:01.544064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:51:01.544064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:51:01.544064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:51:01.544064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:51:01.544064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:51:01.544064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:51:01.544064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:51:01.545103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:51:01.545103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:51:01.545103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:51:01.545103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:51:01.545103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:51:01.545103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:51:01.545103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:51:01.545103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:51:01.545103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:51:01.545103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:51:01.545103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:51:01.545103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:51:01.545103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:51:01.545103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:51:01.545103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:51:01.545103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:51:01.546135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:51:01.546135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:51:01.546135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:51:01.546135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:51:01.546135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:51:01.546135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:51:01.546135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:51:01.546135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:51:01.546135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:51:01.546135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:51:01.546135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:51:01.546135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:51:01.546135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 583/1156] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:51:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:51:01.551499 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbeb84d-9dc9-4e04-9a0d-4b11025c8f19 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:01.595609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:01.647951 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbeb84d-9dc9-4e04-9a0d-4b11025c8f19 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:01.681959 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f3916-d66a-46d8-aa7f-430b7b5e42ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:01.682618 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f3916-d66a-46d8-aa7f-430b7b5e42ec None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:01.682900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f3916-d66a-46d8-aa7f-430b7b5e42ec None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:01.683184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f3916-d66a-46d8-aa7f-430b7b5e42ec None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:01.683934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f3916-d66a-46d8-aa7f-430b7b5e42ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:51:01.684239 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f3916-d66a-46d8-aa7f-430b7b5e42ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:01.685590 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f3916-d66a-46d8-aa7f-430b7b5e42ec None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:01.694116 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 574/1157] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:51:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:51:01.699871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78b9a5-a469-42e1-a2f1-4c67ad7634ad None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:01.799887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78b9a5-a469-42e1-a2f1-4c67ad7634ad None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:01.832141 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8d114-feff-4fdf-8fdd-bc0d6ef5b768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:01.833012 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8d114-feff-4fdf-8fdd-bc0d6ef5b768 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:01.833298 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8d114-feff-4fdf-8fdd-bc0d6ef5b768 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:01.833579 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8d114-feff-4fdf-8fdd-bc0d6ef5b768 None admin] PATH_INFO: `/v3/users/27e2cbd4b28543339e14c4783d92120c` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:01.834760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8d114-feff-4fdf-8fdd-bc0d6ef5b768 None admin] RBAC: Authorizing `identity:get_user(user_id=27e2cbd4b28543339e14c4783d92120c)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:01.836229 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8d114-feff-4fdf-8fdd-bc0d6ef5b768 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:01.837861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 584/1158] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:51:01 2022] GET /identity/v3/users/27e2cbd4b28543339e14c4783d92120c => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:51:01.843907 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f3916-d66a-46d8-aa7f-430b7b5e42ec None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:01.943416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f3916-d66a-46d8-aa7f-430b7b5e42ec None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:01.950199 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:01.976631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab282443-a6de-46bd-9dd2-65a986d78ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:01.977398 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab282443-a6de-46bd-9dd2-65a986d78ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:01.977687 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab282443-a6de-46bd-9dd2-65a986d78ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:01.977966 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab282443-a6de-46bd-9dd2-65a986d78ab1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:02.046615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab282443-a6de-46bd-9dd2-65a986d78ab1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:02.048649 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab282443-a6de-46bd-9dd2-65a986d78ab1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:02.142093 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 575/1159] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:51:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:51:02.148136 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8d114-feff-4fdf-8fdd-bc0d6ef5b768 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:02.199708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:02.244498 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8d114-feff-4fdf-8fdd-bc0d6ef5b768 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:02.277895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c135ea6-28fd-4856-92ff-3cc58699992c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:02.278763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c135ea6-28fd-4856-92ff-3cc58699992c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:02.279058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c135ea6-28fd-4856-92ff-3cc58699992c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:02.279387 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c135ea6-28fd-4856-92ff-3cc58699992c None admin] PATH_INFO: `/v3/projects/f4d2a32ba5f147c5846f28319bfa541b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:02.280595 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c135ea6-28fd-4856-92ff-3cc58699992c None admin] RBAC: Authorizing `identity:get_project(project_id=f4d2a32ba5f147c5846f28319bfa541b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:02.282694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c135ea6-28fd-4856-92ff-3cc58699992c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:02.284445 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 585/1160] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:51:02 2022] GET /identity/v3/projects/f4d2a32ba5f147c5846f28319bfa541b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:51:02.290522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab282443-a6de-46bd-9dd2-65a986d78ab1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:02.298501 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:02.385892 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab282443-a6de-46bd-9dd2-65a986d78ab1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:02.417780 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e8001-ce11-40d0-826c-443cfe1adedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:02.418378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e8001-ce11-40d0-826c-443cfe1adedb None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:02.418666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e8001-ce11-40d0-826c-443cfe1adedb None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:02.418941 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e8001-ce11-40d0-826c-443cfe1adedb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:02.419789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e8001-ce11-40d0-826c-443cfe1adedb None admin] RBAC: Adding query filter params (role.id=27fc14631c494ca0b2dc7f708be016c2, scope.project.id=f4d2a32ba5f147c5846f28319bfa541b, user.id=27e2cbd4b28543339e14c4783d92120c) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:51:02.420102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e8001-ce11-40d0-826c-443cfe1adedb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:02.421520 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e8001-ce11-40d0-826c-443cfe1adedb None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:02.427740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 576/1161] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:51:02 2022] GET /identity/v3/role_assignments?user.id=27e2cbd4b28543339e14c4783d92120c&scope.project.id=f4d2a32ba5f147c5846f28319bfa541b&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:51:04.480308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c710a81-8219-46d2-90e2-138006230681 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:04.480856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c710a81-8219-46d2-90e2-138006230681 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:04.480856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c710a81-8219-46d2-90e2-138006230681 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:04.482259 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 586/1162] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:51:04.487404 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7516ef-2170-4f88-8242-055753caa96b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:04.487802 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7516ef-2170-4f88-8242-055753caa96b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:04.487975 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7516ef-2170-4f88-8242-055753caa96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:04.504305 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7e7516ef-2170-4f88-8242-055753caa96b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:51:04.505478 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7e7516ef-2170-4f88-8242-055753caa96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:51:04.565975 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:04.570398 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 577/1163] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:51:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:51:04.592567 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c710a81-8219-46d2-90e2-138006230681 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:04.692015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c710a81-8219-46d2-90e2-138006230681 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:04.711086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:04.724850 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6a829-6c78-40ee-ae5c-6d3f365c4308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:04.725489 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6a829-6c78-40ee-ae5c-6d3f365c4308 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:04.725775 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6a829-6c78-40ee-ae5c-6d3f365c4308 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:04.726053 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6a829-6c78-40ee-ae5c-6d3f365c4308 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:04.790911 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6a829-6c78-40ee-ae5c-6d3f365c4308 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:04.792332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6a829-6c78-40ee-ae5c-6d3f365c4308 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:04.886453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 587/1164] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:51:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:51:04.892920 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7516ef-2170-4f88-8242-055753caa96b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:04.986136 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7516ef-2170-4f88-8242-055753caa96b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:05.021870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034dc28-efab-4eca-bfd4-2798355327f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:05.022668 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034dc28-efab-4eca-bfd4-2798355327f7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:05.022949 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034dc28-efab-4eca-bfd4-2798355327f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:05.023255 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034dc28-efab-4eca-bfd4-2798355327f7 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:05.029999 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034dc28-efab-4eca-bfd4-2798355327f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:05.031456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034dc28-efab-4eca-bfd4-2798355327f7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:05.045317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-0034dc28-efab-4eca-bfd4-2798355327f7 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 18:51:05.045317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:51:05.045317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 14 18:51:05.045317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 14 18:51:05.045317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:51:05.045317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:51:05.045317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:51:05.045317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:51:05.045317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 14 18:51:05.045317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:51:05.045317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:51:05.045317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:51:05.045317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:51:05.045317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:51:05.045317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:51:05.045317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:51:05.045317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:51:05.046506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:51:05.046506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:51:05.046506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:51:05.046506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:51:05.046506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:51:05.046506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:51:05.046506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:51:05.046506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:51:05.046506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:51:05.046506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:51:05.046506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:51:05.046506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:51:05.046506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:51:05.046506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:51:05.046506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:51:05.046506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:51:05.047405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:51:05.047405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:51:05.047405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:51:05.047405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:51:05.047405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:51:05.047405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:51:05.047405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:51:05.047405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:51:05.047405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:51:05.047405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:51:05.047405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 14 18:51:05.047405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:51:05.047405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 18:51:05.047405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:51:05.047405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 578/1165] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 14 18:51:04 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 18:51:05.054742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6a829-6c78-40ee-ae5c-6d3f365c4308 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:05.075881 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:05.151642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6a829-6c78-40ee-ae5c-6d3f365c4308 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:05.184635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d159f2-bcfc-41e1-937c-a6ecf00ba82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:05.185284 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d159f2-bcfc-41e1-937c-a6ecf00ba82b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:05.185566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d159f2-bcfc-41e1-937c-a6ecf00ba82b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:05.185847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d159f2-bcfc-41e1-937c-a6ecf00ba82b None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:05.186551 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d159f2-bcfc-41e1-937c-a6ecf00ba82b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:51:05.186855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d159f2-bcfc-41e1-937c-a6ecf00ba82b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:05.188809 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d159f2-bcfc-41e1-937c-a6ecf00ba82b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:05.195740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 588/1166] 10.222.0.69 () {56 vars in 1115 bytes} [Thu Jul 14 18:51:05 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:51:07.144351 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88376a8e-0b02-4ee2-8d4e-fc86d845e54d None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:07.144902 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88376a8e-0b02-4ee2-8d4e-fc86d845e54d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:07.144902 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88376a8e-0b02-4ee2-8d4e-fc86d845e54d None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:07.146276 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 579/1167] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:51:07.151474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a65617-08ac-48dc-88ab-66e16d7a84ae None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:07.151889 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a65617-08ac-48dc-88ab-66e16d7a84ae None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:07.152032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a65617-08ac-48dc-88ab-66e16d7a84ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:07.168539 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-34a65617-08ac-48dc-88ab-66e16d7a84ae None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:51:07.169722 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-34a65617-08ac-48dc-88ab-66e16d7a84ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:51:07.232858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 589/1168] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:51:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:51:07.255488 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88376a8e-0b02-4ee2-8d4e-fc86d845e54d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:07.314838 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:07.355763 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88376a8e-0b02-4ee2-8d4e-fc86d845e54d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:07.388011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2598b-77de-4678-bcc2-37594e73f495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:07.388666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2598b-77de-4678-bcc2-37594e73f495 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:07.388951 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2598b-77de-4678-bcc2-37594e73f495 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:07.389239 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2598b-77de-4678-bcc2-37594e73f495 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:07.389981 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2598b-77de-4678-bcc2-37594e73f495 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:07.391982 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2598b-77de-4678-bcc2-37594e73f495 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:07.476423 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 580/1169] 10.222.0.69 () {60 vars in 1138 bytes} [Thu Jul 14 18:51:07 2022] POST /identity/v3/domains => generated 267 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:51:09.487258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc09fac-e5fe-4561-9aaa-8ab8d1b313f1 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:09.487807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc09fac-e5fe-4561-9aaa-8ab8d1b313f1 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:09.487807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc09fac-e5fe-4561-9aaa-8ab8d1b313f1 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:09.489215 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 590/1170] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:51:09.502535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a256040-1057-4df4-886b-97dec5a3493b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:09.502986 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a256040-1057-4df4-886b-97dec5a3493b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:09.503105 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a256040-1057-4df4-886b-97dec5a3493b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:09.519727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3a256040-1057-4df4-886b-97dec5a3493b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:51:09.520920 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3a256040-1057-4df4-886b-97dec5a3493b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:51:09.586445 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 581/1171] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:51:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:51:09.609077 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc09fac-e5fe-4561-9aaa-8ab8d1b313f1 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:09.624858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:09.711865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc09fac-e5fe-4561-9aaa-8ab8d1b313f1 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:09.746433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38561d2-fe04-4615-8aee-5ac660b7c557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:09.747246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38561d2-fe04-4615-8aee-5ac660b7c557 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:09.747674 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38561d2-fe04-4615-8aee-5ac660b7c557 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:09.747961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38561d2-fe04-4615-8aee-5ac660b7c557 None admin] PATH_INFO: `/v3/domains/f3544a351d744c1493d1087b1176e519` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:09.758705 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38561d2-fe04-4615-8aee-5ac660b7c557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f3544a351d744c1493d1087b1176e519)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:09.760217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38561d2-fe04-4615-8aee-5ac660b7c557 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:09.761987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 591/1172] 10.222.0.69 () {56 vars in 1183 bytes} [Thu Jul 14 18:51:09 2022] GET /identity/v3/domains/f3544a351d744c1493d1087b1176e519 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:51:09.768559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a256040-1057-4df4-886b-97dec5a3493b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:09.825024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:09.876401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a256040-1057-4df4-886b-97dec5a3493b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:09.912019 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42c087-0fcd-4cca-936e-6002f03351f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:09.912688 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42c087-0fcd-4cca-936e-6002f03351f1 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:09.912981 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42c087-0fcd-4cca-936e-6002f03351f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:09.913266 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42c087-0fcd-4cca-936e-6002f03351f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:09.913989 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42c087-0fcd-4cca-936e-6002f03351f1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:09.915470 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42c087-0fcd-4cca-936e-6002f03351f1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:09.996616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 582/1173] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 14 18:51:09 2022] POST /identity/v3/projects => generated 361 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:51:12.008434 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791200f6-cbf5-44f4-8682-5a2d3b5f416b None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:12.008898 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791200f6-cbf5-44f4-8682-5a2d3b5f416b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:12.009012 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791200f6-cbf5-44f4-8682-5a2d3b5f416b None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:12.010401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 592/1174] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:51:12.015761 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e24bc-a24e-49e6-905a-3250f9c94433 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:12.016684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e24bc-a24e-49e6-905a-3250f9c94433 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:12.016830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e24bc-a24e-49e6-905a-3250f9c94433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:12.033964 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-fc9e24bc-a24e-49e6-905a-3250f9c94433 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:51:12.035130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-fc9e24bc-a24e-49e6-905a-3250f9c94433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:51:12.100002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 583/1175] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:51:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 18:51:12.124760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791200f6-cbf5-44f4-8682-5a2d3b5f416b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:12.214112 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:12.238730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791200f6-cbf5-44f4-8682-5a2d3b5f416b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:12.272125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62505b79-5369-4138-9056-f841c9eb2600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:12.272936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62505b79-5369-4138-9056-f841c9eb2600 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:12.273220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62505b79-5369-4138-9056-f841c9eb2600 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:12.273497 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62505b79-5369-4138-9056-f841c9eb2600 None admin] PATH_INFO: `/v3/domains/f3544a351d744c1493d1087b1176e519` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:12.274667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62505b79-5369-4138-9056-f841c9eb2600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f3544a351d744c1493d1087b1176e519)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:12.276143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62505b79-5369-4138-9056-f841c9eb2600 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:12.278030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 593/1176] 10.222.0.69 () {56 vars in 1183 bytes} [Thu Jul 14 18:51:12 2022] GET /identity/v3/domains/f3544a351d744c1493d1087b1176e519 => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:51:12.284579 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e24bc-a24e-49e6-905a-3250f9c94433 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:12.314410 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:12.380243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e24bc-a24e-49e6-905a-3250f9c94433 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:12.412783 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de7fab-cde5-45a7-b32f-e69ae5b3ce29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:12.413435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de7fab-cde5-45a7-b32f-e69ae5b3ce29 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:12.413716 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de7fab-cde5-45a7-b32f-e69ae5b3ce29 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:12.413988 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de7fab-cde5-45a7-b32f-e69ae5b3ce29 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:12.414692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de7fab-cde5-45a7-b32f-e69ae5b3ce29 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:12.416139 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de7fab-cde5-45a7-b32f-e69ae5b3ce29 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:12.506027 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 584/1177] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 14 18:51:12 2022] POST /identity/v3/users => generated 311 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:51:14.511186 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1eb32-776c-44bc-933e-ab49dc3ba258 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:14.511744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1eb32-776c-44bc-933e-ab49dc3ba258 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:14.511744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1eb32-776c-44bc-933e-ab49dc3ba258 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:14.513175 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 594/1178] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:51:14.518243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323a777-39bd-4c16-9769-5742cfa5aea3 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:14.518656 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323a777-39bd-4c16-9769-5742cfa5aea3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:14.518811 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323a777-39bd-4c16-9769-5742cfa5aea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:14.536081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a323a777-39bd-4c16-9769-5742cfa5aea3 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:51:14.537271 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a323a777-39bd-4c16-9769-5742cfa5aea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:51:14.602129 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 585/1179] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:51:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:51:14.625404 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1eb32-776c-44bc-933e-ab49dc3ba258 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:14.723023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1eb32-776c-44bc-933e-ab49dc3ba258 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:14.755665 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841772f-83ae-462d-af20-46b7f829b5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:14.756440 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841772f-83ae-462d-af20-46b7f829b5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:14.756726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841772f-83ae-462d-af20-46b7f829b5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:14.757003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841772f-83ae-462d-af20-46b7f829b5d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:14.766857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841772f-83ae-462d-af20-46b7f829b5d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:51:14.774372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-7841772f-83ae-462d-af20-46b7f829b5d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:51:14.774372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:51:14.774372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:51:14.774372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:51:14.774372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:51:14.774372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:51:14.774372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:51:14.774372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:51:14.774372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:51:14.774372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:51:14.774372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:51:14.774372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:51:14.774372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:51:14.774372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:51:14.774372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:51:14.774372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:51:14.775467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:51:14.775467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:51:14.775467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:51:14.775467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:51:14.775467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:51:14.775467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:51:14.775467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:51:14.775467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:51:14.775467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:51:14.775467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:51:14.775467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:51:14.775467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:51:14.775467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:51:14.775467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:51:14.775467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:51:14.775467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:51:14.776356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:51:14.776356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:51:14.776356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:51:14.776356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:51:14.776356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:51:14.776356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:51:14.776356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:51:14.776356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:51:14.776356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:51:14.776356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:51:14.776356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:51:14.776356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:51:14.776356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 595/1180] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:51:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:51:14.782201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323a777-39bd-4c16-9769-5742cfa5aea3 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:14.803201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:14.880916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323a777-39bd-4c16-9769-5742cfa5aea3 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:14.914054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6567066-ebbf-4391-86b0-af37c279eb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:14.914716 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6567066-ebbf-4391-86b0-af37c279eb6f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:14.915005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6567066-ebbf-4391-86b0-af37c279eb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:14.915319 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6567066-ebbf-4391-86b0-af37c279eb6f None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:14.916063 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6567066-ebbf-4391-86b0-af37c279eb6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:51:14.916376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6567066-ebbf-4391-86b0-af37c279eb6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:14.917752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6567066-ebbf-4391-86b0-af37c279eb6f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:14.926265 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 586/1181] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:51:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:51:14.932117 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841772f-83ae-462d-af20-46b7f829b5d3 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:14.934941 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:15.029868 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841772f-83ae-462d-af20-46b7f829b5d3 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:15.062029 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ce9ec-9958-47bf-8751-db33040da1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:15.062916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ce9ec-9958-47bf-8751-db33040da1ce None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:15.063221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ce9ec-9958-47bf-8751-db33040da1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:15.063502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ce9ec-9958-47bf-8751-db33040da1ce None admin] PATH_INFO: `/v3/users/eecfe99c8ce24d339c7b9afede093772` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:15.076059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ce9ec-9958-47bf-8751-db33040da1ce None admin] RBAC: Authorizing `identity:get_user(user_id=eecfe99c8ce24d339c7b9afede093772)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:15.077508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ce9ec-9958-47bf-8751-db33040da1ce None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:15.079722 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 596/1182] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:51:14 2022] GET /identity/v3/users/eecfe99c8ce24d339c7b9afede093772 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:51:15.085497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6567066-ebbf-4391-86b0-af37c279eb6f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:15.165016 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:15.182963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6567066-ebbf-4391-86b0-af37c279eb6f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:15.216066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6a654-a15c-4864-8457-5d42495790dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:15.216701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6a654-a15c-4864-8457-5d42495790dd None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:15.216988 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6a654-a15c-4864-8457-5d42495790dd None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:15.217265 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6a654-a15c-4864-8457-5d42495790dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:15.282223 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6a654-a15c-4864-8457-5d42495790dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:15.283644 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6a654-a15c-4864-8457-5d42495790dd None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:15.359342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:15.378233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 587/1183] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:51:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:51:15.384239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ce9ec-9958-47bf-8751-db33040da1ce None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:15.441425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:15.479570 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ce9ec-9958-47bf-8751-db33040da1ce None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:15.512265 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbaa0a-d7d7-473c-a4c4-c4e3d416b763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:15.513122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbaa0a-d7d7-473c-a4c4-c4e3d416b763 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:15.513409 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbaa0a-d7d7-473c-a4c4-c4e3d416b763 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:15.513683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbaa0a-d7d7-473c-a4c4-c4e3d416b763 None admin] PATH_INFO: `/v3/projects/77f66311bb294d9c8aed3c60f2e907d4` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:15.514852 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbaa0a-d7d7-473c-a4c4-c4e3d416b763 None admin] RBAC: Authorizing `identity:get_project(project_id=77f66311bb294d9c8aed3c60f2e907d4)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:15.516880 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbaa0a-d7d7-473c-a4c4-c4e3d416b763 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:15.518562 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 597/1184] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:51:15 2022] GET /identity/v3/projects/77f66311bb294d9c8aed3c60f2e907d4 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:51:15.524865 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6a654-a15c-4864-8457-5d42495790dd None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:15.620033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6a654-a15c-4864-8457-5d42495790dd None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:15.653651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec1b97-c672-4969-ad41-c3421270339b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:15.654298 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec1b97-c672-4969-ad41-c3421270339b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:15.654589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec1b97-c672-4969-ad41-c3421270339b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:15.654868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec1b97-c672-4969-ad41-c3421270339b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:15.655720 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec1b97-c672-4969-ad41-c3421270339b None admin] RBAC: Adding query filter params (role.id=27fc14631c494ca0b2dc7f708be016c2, scope.project.id=77f66311bb294d9c8aed3c60f2e907d4, user.id=eecfe99c8ce24d339c7b9afede093772) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:51:15.656038 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec1b97-c672-4969-ad41-c3421270339b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:15.657570 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec1b97-c672-4969-ad41-c3421270339b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:15.662862 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 588/1185] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:51:15 2022] GET /identity/v3/role_assignments?user.id=eecfe99c8ce24d339c7b9afede093772&scope.project.id=77f66311bb294d9c8aed3c60f2e907d4&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:51:17.620062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cb865-4e06-4832-8e83-3a301ceb1f9a None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:17.620605 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cb865-4e06-4832-8e83-3a301ceb1f9a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:17.620605 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cb865-4e06-4832-8e83-3a301ceb1f9a None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:17.621991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 598/1186] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:51:17.627629 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422b91c-adbf-4f1d-a177-5b9240bfe898 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:17.628000 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422b91c-adbf-4f1d-a177-5b9240bfe898 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:17.628188 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422b91c-adbf-4f1d-a177-5b9240bfe898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:17.645250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-e422b91c-adbf-4f1d-a177-5b9240bfe898 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:51:17.646427 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e422b91c-adbf-4f1d-a177-5b9240bfe898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:51:17.684446 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:17.710321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 589/1187] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:51:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:51:17.732124 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699cb865-4e06-4832-8e83-3a301ceb1f9a None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:17.829823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699cb865-4e06-4832-8e83-3a301ceb1f9a None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:17.859781 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:17.867184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6d3ee-2b34-4814-a57d-39d137eb1726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:17.870712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6d3ee-2b34-4814-a57d-39d137eb1726 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:17.870712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6d3ee-2b34-4814-a57d-39d137eb1726 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:17.870712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6d3ee-2b34-4814-a57d-39d137eb1726 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:17.878870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6d3ee-2b34-4814-a57d-39d137eb1726 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:51:17.886685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-b3e6d3ee-2b34-4814-a57d-39d137eb1726 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:51:17.886685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:51:17.886685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:51:17.886685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:51:17.886685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:51:17.886685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:51:17.886685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:51:17.886685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:51:17.886685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:51:17.886685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:51:17.886685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:51:17.886685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:51:17.886685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:51:17.886685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:51:17.886685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:51:17.886685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:51:17.887615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:51:17.887615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:51:17.887615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:51:17.887615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:51:17.887615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:51:17.887615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:51:17.887615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:51:17.887615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:51:17.887615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:51:17.887615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:51:17.887615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:51:17.887615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:51:17.887615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:51:17.887615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:51:17.887615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:51:17.887615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:51:17.888532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:51:17.888532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:51:17.888532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:51:17.888532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:51:17.888532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:51:17.888532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:51:17.888532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:51:17.888532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:51:17.888532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:51:17.888532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:51:17.888532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:51:17.888532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:51:17.888532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 599/1188] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:51:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:51:17.894806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422b91c-adbf-4f1d-a177-5b9240bfe898 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:17.990320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422b91c-adbf-4f1d-a177-5b9240bfe898 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:18.023879 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62567c7-e242-4627-bd70-5a42bb49554c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:18.024559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62567c7-e242-4627-bd70-5a42bb49554c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:18.024851 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62567c7-e242-4627-bd70-5a42bb49554c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:18.025131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62567c7-e242-4627-bd70-5a42bb49554c None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:18.025874 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62567c7-e242-4627-bd70-5a42bb49554c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:51:18.026182 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62567c7-e242-4627-bd70-5a42bb49554c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:18.027833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62567c7-e242-4627-bd70-5a42bb49554c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:18.036648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 590/1189] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:51:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:51:18.042486 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6d3ee-2b34-4814-a57d-39d137eb1726 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:18.151823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6d3ee-2b34-4814-a57d-39d137eb1726 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:18.185040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58a767-d521-406b-b7b7-45938fb7c5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:18.185908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58a767-d521-406b-b7b7-45938fb7c5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:18.186195 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58a767-d521-406b-b7b7-45938fb7c5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:18.186469 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58a767-d521-406b-b7b7-45938fb7c5f2 None admin] PATH_INFO: `/v3/users/eecfe99c8ce24d339c7b9afede093772` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:18.187669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58a767-d521-406b-b7b7-45938fb7c5f2 None admin] RBAC: Authorizing `identity:get_user(user_id=eecfe99c8ce24d339c7b9afede093772)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:18.189121 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58a767-d521-406b-b7b7-45938fb7c5f2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:18.190756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 600/1190] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:51:18 2022] GET /identity/v3/users/eecfe99c8ce24d339c7b9afede093772 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:51:18.196554 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62567c7-e242-4627-bd70-5a42bb49554c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:18.232103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:18.293782 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62567c7-e242-4627-bd70-5a42bb49554c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:18.326643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695a011-96ab-49c2-83f0-9821e0c47979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:18.327516 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695a011-96ab-49c2-83f0-9821e0c47979 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:18.327803 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695a011-96ab-49c2-83f0-9821e0c47979 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:18.328079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695a011-96ab-49c2-83f0-9821e0c47979 None admin] PATH_INFO: `/v3/projects/77f66311bb294d9c8aed3c60f2e907d4` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:18.329842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695a011-96ab-49c2-83f0-9821e0c47979 None admin] RBAC: Authorizing `identity:get_project(project_id=77f66311bb294d9c8aed3c60f2e907d4)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:18.331295 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695a011-96ab-49c2-83f0-9821e0c47979 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:18.332972 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 591/1191] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:51:18 2022] GET /identity/v3/projects/77f66311bb294d9c8aed3c60f2e907d4 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:51:18.338782 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58a767-d521-406b-b7b7-45938fb7c5f2 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:18.393750 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:18.436058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58a767-d521-406b-b7b7-45938fb7c5f2 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:18.469046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891a0b6-eae0-419f-92f9-6c9853b6a08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:18.469924 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891a0b6-eae0-419f-92f9-6c9853b6a08d None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:18.470207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891a0b6-eae0-419f-92f9-6c9853b6a08d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:18.470481 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891a0b6-eae0-419f-92f9-6c9853b6a08d None admin] PATH_INFO: `/v3/projects/77f66311bb294d9c8aed3c60f2e907d4/users/eecfe99c8ce24d339c7b9afede093772/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:18.472411 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891a0b6-eae0-419f-92f9-6c9853b6a08d None admin] RBAC: Authorizing `identity:create_grant(project_id=77f66311bb294d9c8aed3c60f2e907d4, user_id=eecfe99c8ce24d339c7b9afede093772, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:18.473864 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891a0b6-eae0-419f-92f9-6c9853b6a08d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:18.559093 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 601/1192] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:51:18 2022] PUT /identity/v3/projects/77f66311bb294d9c8aed3c60f2e907d4/users/eecfe99c8ce24d339c7b9afede093772/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 18:51:20.522002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51110d-dc3a-4c50-8a47-de942469869b None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:20.522571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51110d-dc3a-4c50-8a47-de942469869b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:20.522571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51110d-dc3a-4c50-8a47-de942469869b None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:20.523957 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 592/1193] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:51:20.529378 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb63e97-cb09-4c46-b9e1-8f90670a151e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:20.529788 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb63e97-cb09-4c46-b9e1-8f90670a151e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:20.529940 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb63e97-cb09-4c46-b9e1-8f90670a151e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:20.546726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9fb63e97-cb09-4c46-b9e1-8f90670a151e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:51:20.548529 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9fb63e97-cb09-4c46-b9e1-8f90670a151e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:51:20.611791 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 602/1194] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:51:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:51:20.633975 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51110d-dc3a-4c50-8a47-de942469869b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:20.732624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51110d-dc3a-4c50-8a47-de942469869b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:20.735507 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:20.765429 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b8a31-25fd-4203-868a-c992ae646e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:20.766435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b8a31-25fd-4203-868a-c992ae646e38 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:20.766723 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b8a31-25fd-4203-868a-c992ae646e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:20.767002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b8a31-25fd-4203-868a-c992ae646e38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:20.777101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b8a31-25fd-4203-868a-c992ae646e38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:51:20.785936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-6a0b8a31-25fd-4203-868a-c992ae646e38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:51:20.785936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:51:20.785936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:51:20.785936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:51:20.785936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:51:20.785936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:51:20.785936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:51:20.785936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:51:20.785936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:51:20.785936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:51:20.785936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:51:20.785936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:51:20.785936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:51:20.785936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:51:20.785936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:51:20.785936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:51:20.787084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:51:20.787084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:51:20.787084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:51:20.787084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:51:20.787084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:51:20.787084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:51:20.787084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:51:20.787084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:51:20.787084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:51:20.787084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:51:20.787084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:51:20.787084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:51:20.787084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:51:20.787084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:51:20.787084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:51:20.787084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:51:20.788160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:51:20.788160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:51:20.788160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:51:20.788160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:51:20.788160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:51:20.788160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:51:20.788160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:51:20.788160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:51:20.788160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:51:20.788160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:51:20.788160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:51:20.788160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:51:20.788160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 593/1195] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:51:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:51:20.793304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb63e97-cb09-4c46-b9e1-8f90670a151e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:20.838427 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:20.890279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb63e97-cb09-4c46-b9e1-8f90670a151e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:20.924763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262e54a-f7b3-4b79-8ed9-0e82bd283143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:20.925424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262e54a-f7b3-4b79-8ed9-0e82bd283143 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:20.925717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262e54a-f7b3-4b79-8ed9-0e82bd283143 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:20.926001 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262e54a-f7b3-4b79-8ed9-0e82bd283143 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:20.926747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262e54a-f7b3-4b79-8ed9-0e82bd283143 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:51:20.927059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262e54a-f7b3-4b79-8ed9-0e82bd283143 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:20.928473 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262e54a-f7b3-4b79-8ed9-0e82bd283143 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:20.936302 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 603/1196] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:51:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:51:20.941920 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b8a31-25fd-4203-868a-c992ae646e38 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:21.038763 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b8a31-25fd-4203-868a-c992ae646e38 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:21.070153 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b0cf9-79f5-4104-a28f-48a0184ca338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:21.071010 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b0cf9-79f5-4104-a28f-48a0184ca338 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:21.071476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b0cf9-79f5-4104-a28f-48a0184ca338 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:21.071783 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b0cf9-79f5-4104-a28f-48a0184ca338 None admin] PATH_INFO: `/v3/users/eecfe99c8ce24d339c7b9afede093772` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:21.072956 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b0cf9-79f5-4104-a28f-48a0184ca338 None admin] RBAC: Authorizing `identity:get_user(user_id=eecfe99c8ce24d339c7b9afede093772)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:21.074370 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b0cf9-79f5-4104-a28f-48a0184ca338 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:21.075996 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 594/1197] 10.222.0.69 () {56 vars in 1177 bytes} [Thu Jul 14 18:51:20 2022] GET /identity/v3/users/eecfe99c8ce24d339c7b9afede093772 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:51:21.082182 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262e54a-f7b3-4b79-8ed9-0e82bd283143 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:21.177959 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262e54a-f7b3-4b79-8ed9-0e82bd283143 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:21.186334 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:21.210538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13360068-40cd-4ad3-8ff8-409bba227a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:21.211234 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13360068-40cd-4ad3-8ff8-409bba227a1d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:21.211530 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13360068-40cd-4ad3-8ff8-409bba227a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:21.211807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13360068-40cd-4ad3-8ff8-409bba227a1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:21.276231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13360068-40cd-4ad3-8ff8-409bba227a1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:21.277924 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13360068-40cd-4ad3-8ff8-409bba227a1d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:21.371935 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 604/1198] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:51:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:51:21.378225 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b0cf9-79f5-4104-a28f-48a0184ca338 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:21.409083 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:21.475148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b0cf9-79f5-4104-a28f-48a0184ca338 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:21.507993 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0312f-72f1-4d44-b589-366f657e85ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:21.508844 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0312f-72f1-4d44-b589-366f657e85ef None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:21.509131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0312f-72f1-4d44-b589-366f657e85ef None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:21.509411 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0312f-72f1-4d44-b589-366f657e85ef None admin] PATH_INFO: `/v3/projects/77f66311bb294d9c8aed3c60f2e907d4` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:21.510575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0312f-72f1-4d44-b589-366f657e85ef None admin] RBAC: Authorizing `identity:get_project(project_id=77f66311bb294d9c8aed3c60f2e907d4)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:21.512037 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0312f-72f1-4d44-b589-366f657e85ef None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:21.513722 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 595/1199] 10.222.0.69 () {56 vars in 1186 bytes} [Thu Jul 14 18:51:21 2022] GET /identity/v3/projects/77f66311bb294d9c8aed3c60f2e907d4 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:51:21.520288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13360068-40cd-4ad3-8ff8-409bba227a1d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:51:21.530555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:21.616345 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13360068-40cd-4ad3-8ff8-409bba227a1d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:51:21.650088 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889b4d1-0380-4976-8dd6-182879d8e50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:51:21.650680 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889b4d1-0380-4976-8dd6-182879d8e50f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:51:21.650966 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889b4d1-0380-4976-8dd6-182879d8e50f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:51:21.651270 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889b4d1-0380-4976-8dd6-182879d8e50f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:51:21.652091 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889b4d1-0380-4976-8dd6-182879d8e50f None admin] RBAC: Adding query filter params (role.id=27fc14631c494ca0b2dc7f708be016c2, scope.project.id=77f66311bb294d9c8aed3c60f2e907d4, user.id=eecfe99c8ce24d339c7b9afede093772) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:51:21.652667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889b4d1-0380-4976-8dd6-182879d8e50f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:51:21.654105 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889b4d1-0380-4976-8dd6-182879d8e50f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:51:21.660283 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 605/1200] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:51:21 2022] GET /identity/v3/role_assignments?user.id=eecfe99c8ce24d339c7b9afede093772&scope.project.id=77f66311bb294d9c8aed3c60f2e907d4&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:51:31.503906 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=50 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:51:31.657694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=64 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:33.367609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532cb203-4f09-4be6-8e75-f0e7119616fa None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:33.368200 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532cb203-4f09-4be6-8e75-f0e7119616fa None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:33.368200 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532cb203-4f09-4be6-8e75-f0e7119616fa None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:33.369548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 596/1201] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:52:33.374575 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67710aae-918a-496c-901a-d016e91e47b6 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:33.375034 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67710aae-918a-496c-901a-d016e91e47b6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:33.375139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67710aae-918a-496c-901a-d016e91e47b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:33.392874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-67710aae-918a-496c-901a-d016e91e47b6 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:52:33.394219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-67710aae-918a-496c-901a-d016e91e47b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:52:33.458887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 606/1202] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:52:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:52:33.482559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532cb203-4f09-4be6-8e75-f0e7119616fa None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:33.584025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532cb203-4f09-4be6-8e75-f0e7119616fa None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:33.617089 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9c331-bce4-4b86-94e4-b954ad377a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:33.617919 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9c331-bce4-4b86-94e4-b954ad377a7d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:33.618155 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9c331-bce4-4b86-94e4-b954ad377a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:33.618429 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9c331-bce4-4b86-94e4-b954ad377a7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:33.628882 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9c331-bce4-4b86-94e4-b954ad377a7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:33.630321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9c331-bce4-4b86-94e4-b954ad377a7d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:33.648321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-05c9c331-bce4-4b86-94e4-b954ad377a7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:33.648321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:33.648321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:33.648321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:33.648321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:33.648321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:33.648321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:33.648321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:33.648321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:33.648321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:33.648321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:33.648321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:33.648321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:52:33.648321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:52:33.648321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:52:33.649471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:52:33.649471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:33.649471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:33.649471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:33.649471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:33.649471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:33.649471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:33.649471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:33.649471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:33.649471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:33.649471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:33.649471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:33.649471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:33.649471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:33.649471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:33.649471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:33.649471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:33.650537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:52:33.650537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:52:33.650537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:33.650537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:52:33.650537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 597/1203] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:33.655852 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67710aae-918a-496c-901a-d016e91e47b6 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:33.753334 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67710aae-918a-496c-901a-d016e91e47b6 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:33.785203 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:33.790964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83df27d-0ca4-4f97-9694-8cc57a13e70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:33.791672 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83df27d-0ca4-4f97-9694-8cc57a13e70f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:33.791965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83df27d-0ca4-4f97-9694-8cc57a13e70f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:33.792248 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83df27d-0ca4-4f97-9694-8cc57a13e70f None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:33.792965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83df27d-0ca4-4f97-9694-8cc57a13e70f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:33.793277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83df27d-0ca4-4f97-9694-8cc57a13e70f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:33.794664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83df27d-0ca4-4f97-9694-8cc57a13e70f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:33.806210 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 607/1204] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:33.812188 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9c331-bce4-4b86-94e4-b954ad377a7d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:33.866640 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:33.912497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9c331-bce4-4b86-94e4-b954ad377a7d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:33.947124 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e578f78-1a01-4856-9f3c-316703764edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:33.947888 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e578f78-1a01-4856-9f3c-316703764edd None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:33.948180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e578f78-1a01-4856-9f3c-316703764edd None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:33.948464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e578f78-1a01-4856-9f3c-316703764edd None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:33.949197 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e578f78-1a01-4856-9f3c-316703764edd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:33.950627 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e578f78-1a01-4856-9f3c-316703764edd None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:34.065333 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 598/1205] 10.222.0.69 () {60 vars in 1132 bytes} [Thu Jul 14 18:52:33 2022] POST /identity/v3/users => generated 251 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:52:36.085908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e1c2e-4f1e-4f00-834d-cb1452a79cfd None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:36.086508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e1c2e-4f1e-4f00-834d-cb1452a79cfd None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:36.086508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e1c2e-4f1e-4f00-834d-cb1452a79cfd None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:36.087930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 608/1206] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:52:36.093306 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74410422-7122-45c3-b429-028c55b3ec67 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:36.093720 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74410422-7122-45c3-b429-028c55b3ec67 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:36.093896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74410422-7122-45c3-b429-028c55b3ec67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:36.111080 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-74410422-7122-45c3-b429-028c55b3ec67 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:52:36.112308 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-74410422-7122-45c3-b429-028c55b3ec67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:52:36.178601 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 599/1207] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:52:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:52:36.206692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e1c2e-4f1e-4f00-834d-cb1452a79cfd None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:36.307029 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e1c2e-4f1e-4f00-834d-cb1452a79cfd None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:36.341179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a5b4f-b966-4705-a9df-0214e5ed432c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:36.342020 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a5b4f-b966-4705-a9df-0214e5ed432c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:36.342306 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a5b4f-b966-4705-a9df-0214e5ed432c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:36.342586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a5b4f-b966-4705-a9df-0214e5ed432c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:36.352850 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a5b4f-b966-4705-a9df-0214e5ed432c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:52:36.360768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-3b1a5b4f-b966-4705-a9df-0214e5ed432c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:52:36.360768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:36.360768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:36.360768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:36.360768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:36.360768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:36.360768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:36.360768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:36.360768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:36.360768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:36.360768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:36.360768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:36.360768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:52:36.360768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:52:36.360768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:52:36.360768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:52:36.361865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:52:36.361865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:52:36.361865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:52:36.361865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:52:36.361865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:36.361865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:36.361865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:36.361865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:36.361865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:36.361865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:36.361865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:36.361865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:36.361865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:36.361865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:36.361865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:36.361865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:36.362909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:36.362909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:36.362909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:36.362909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:36.362909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:52:36.362909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:52:36.362909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:52:36.362909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:52:36.362909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:52:36.362909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:52:36.362909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:52:36.362909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:52:36.362909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 609/1208] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:52:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:36.368250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74410422-7122-45c3-b429-028c55b3ec67 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:36.409622 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:36.466611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74410422-7122-45c3-b429-028c55b3ec67 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:36.499898 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8998e3f-559a-496b-8f97-8836c37980e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:36.500555 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8998e3f-559a-496b-8f97-8836c37980e9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:36.500840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8998e3f-559a-496b-8f97-8836c37980e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:36.501643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8998e3f-559a-496b-8f97-8836c37980e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:36.502376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8998e3f-559a-496b-8f97-8836c37980e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:36.502685 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8998e3f-559a-496b-8f97-8836c37980e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:36.504108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8998e3f-559a-496b-8f97-8836c37980e9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:36.521085 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 600/1209] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:52:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:36.527073 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a5b4f-b966-4705-a9df-0214e5ed432c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:36.576363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:36.625419 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a5b4f-b966-4705-a9df-0214e5ed432c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:36.658220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0fc79-4a32-485f-8887-99c5ab208fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:36.659010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0fc79-4a32-485f-8887-99c5ab208fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:36.659339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0fc79-4a32-485f-8887-99c5ab208fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:36.659620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0fc79-4a32-485f-8887-99c5ab208fe5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:36.671960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0fc79-4a32-485f-8887-99c5ab208fe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:36.674019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0fc79-4a32-485f-8887-99c5ab208fe5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:36.691502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-e8f0fc79-4a32-485f-8887-99c5ab208fe5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:36.691502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:36.691502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:36.691502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:36.691502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:36.691502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:36.691502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:36.691502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:36.691502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:36.691502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:36.691502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:36.691502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:36.691502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:52:36.691502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:52:36.691502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:52:36.692559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:52:36.692559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:36.692559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:36.692559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:36.692559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:36.692559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:36.692559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:36.692559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:36.692559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:36.692559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:36.692559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:36.692559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:36.692559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:36.692559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:36.692559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:36.692559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:36.692559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:36.693656 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:52:36.693656 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:52:36.693656 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:36.693656 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:52:36.693656 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 610/1210] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:36.698404 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8998e3f-559a-496b-8f97-8836c37980e9 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:36.793486 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:36.796384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8998e3f-559a-496b-8f97-8836c37980e9 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:36.850640 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d68bb0-2521-4713-be93-2c10ebc942eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:36.851255 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d68bb0-2521-4713-be93-2c10ebc942eb None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:36.851542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d68bb0-2521-4713-be93-2c10ebc942eb None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:36.851820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d68bb0-2521-4713-be93-2c10ebc942eb None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:36.852619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d68bb0-2521-4713-be93-2c10ebc942eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:36.852945 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d68bb0-2521-4713-be93-2c10ebc942eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:36.854334 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d68bb0-2521-4713-be93-2c10ebc942eb None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:36.867691 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 601/1211] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:36.873401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0fc79-4a32-485f-8887-99c5ab208fe5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:36.974054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0fc79-4a32-485f-8887-99c5ab208fe5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:36.975589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:37.007612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392e64c-6992-451a-86e6-7c1e978712c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:37.008427 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392e64c-6992-451a-86e6-7c1e978712c6 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:37.008716 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392e64c-6992-451a-86e6-7c1e978712c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:37.008994 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392e64c-6992-451a-86e6-7c1e978712c6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:37.016546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4392e64c-6992-451a-86e6-7c1e978712c6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:37.018032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4392e64c-6992-451a-86e6-7c1e978712c6 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:37.033038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-4392e64c-6992-451a-86e6-7c1e978712c6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 18:52:37.033038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:37.033038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:37.033038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:37.033038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:37.033038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:37.033038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:37.033038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:37.033038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:37.033038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:37.033038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:37.033038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:37.033038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:52:37.033038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:52:37.033038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:52:37.033038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:52:37.034174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:37.034174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:37.034174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:52:37.034174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:52:37.034174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:52:37.034174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:52:37.034174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:37.034174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:37.034174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:37.034174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:37.034174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:37.034174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:37.034174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:37.034174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:37.034174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:37.034174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:37.034174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:37.034174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:37.035475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:37.035475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:37.035475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:52:37.035475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:52:37.035475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:52:37.035475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:52:37.035475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:52:37.035475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:52:37.035475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 18:52:37.035475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:52:37.035475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 611/1212] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:37.041066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d68bb0-2521-4713-be93-2c10ebc942eb None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:37.140926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d68bb0-2521-4713-be93-2c10ebc942eb None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:37.176900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ca6e2-918c-4c66-9a1e-8446f4d785ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:37.177570 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ca6e2-918c-4c66-9a1e-8446f4d785ed None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:37.177857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ca6e2-918c-4c66-9a1e-8446f4d785ed None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:37.178132 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ca6e2-918c-4c66-9a1e-8446f4d785ed None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:37.178961 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ca6e2-918c-4c66-9a1e-8446f4d785ed None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:37.179301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ca6e2-918c-4c66-9a1e-8446f4d785ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:37.180729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ca6e2-918c-4c66-9a1e-8446f4d785ed None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:37.198300 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 602/1213] 10.222.0.69 () {56 vars in 1143 bytes} [Thu Jul 14 18:52:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:37.204216 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392e64c-6992-451a-86e6-7c1e978712c6 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:37.302853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392e64c-6992-451a-86e6-7c1e978712c6 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:37.335439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09897607-2430-4d18-8b82-092a5742969a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:37.336079 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09897607-2430-4d18-8b82-092a5742969a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:37.336364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09897607-2430-4d18-8b82-092a5742969a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:37.336642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09897607-2430-4d18-8b82-092a5742969a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:37.365629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:37.401816 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09897607-2430-4d18-8b82-092a5742969a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:37.403480 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09897607-2430-4d18-8b82-092a5742969a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:37.498441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 612/1214] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:52:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:52:37.504301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ca6e2-918c-4c66-9a1e-8446f4d785ed None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:37.523292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:37.604926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ca6e2-918c-4c66-9a1e-8446f4d785ed None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:37.637598 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72e056-196c-4826-9f97-27e8e9d135d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:37.638382 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72e056-196c-4826-9f97-27e8e9d135d6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:37.638668 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72e056-196c-4826-9f97-27e8e9d135d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:37.638942 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72e056-196c-4826-9f97-27e8e9d135d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:37.649628 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72e056-196c-4826-9f97-27e8e9d135d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:37.651087 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72e056-196c-4826-9f97-27e8e9d135d6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:37.669407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-8d72e056-196c-4826-9f97-27e8e9d135d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:37.669407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:37.669407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:37.669407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:37.669407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:37.669407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:37.669407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:37.669407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:37.669407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:37.669407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:37.669407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:37.669407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:37.669407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:52:37.669407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:52:37.669407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:52:37.670535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:52:37.670535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:37.670535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:37.670535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:37.670535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:37.670535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:37.670535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:37.670535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:37.670535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:37.670535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:37.670535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:37.670535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:37.670535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:37.670535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:37.670535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:37.670535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:37.670535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:37.671448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:52:37.671448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:52:37.671448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:37.671448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:52:37.671448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 603/1215] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:37.676279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09897607-2430-4d18-8b82-092a5742969a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:37.734675 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:37.771583 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09897607-2430-4d18-8b82-092a5742969a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:37.803098 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16a36b-c93e-4ba6-84dc-d4c688771386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:37.803783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16a36b-c93e-4ba6-84dc-d4c688771386 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:37.804065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16a36b-c93e-4ba6-84dc-d4c688771386 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:37.804338 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16a36b-c93e-4ba6-84dc-d4c688771386 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:37.805046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16a36b-c93e-4ba6-84dc-d4c688771386 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:37.805350 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16a36b-c93e-4ba6-84dc-d4c688771386 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:37.806727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16a36b-c93e-4ba6-84dc-d4c688771386 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:37.819455 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 613/1216] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:37.825315 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72e056-196c-4826-9f97-27e8e9d135d6 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:37.895416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:37.927539 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72e056-196c-4826-9f97-27e8e9d135d6 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:37.960536 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307a444-c15a-4fbf-aa4b-84bcbac97919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:37.961485 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307a444-c15a-4fbf-aa4b-84bcbac97919 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:37.961808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307a444-c15a-4fbf-aa4b-84bcbac97919 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:37.962480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307a444-c15a-4fbf-aa4b-84bcbac97919 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:37.969018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307a444-c15a-4fbf-aa4b-84bcbac97919 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:37.970649 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307a444-c15a-4fbf-aa4b-84bcbac97919 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:37.984513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-0307a444-c15a-4fbf-aa4b-84bcbac97919 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:52:37.984513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:37.984513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:37.984513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:37.984513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:37.984513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:37.984513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:37.984513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:37.984513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:37.984513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:37.984513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:37.984513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:37.984513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:52:37.984513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:52:37.984513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:52:37.985544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:52:37.985544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:37.985544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:37.985544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:37.985544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:37.985544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:37.985544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:37.985544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:37.985544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:37.985544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:37.985544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:37.985544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:37.985544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:37.985544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:37.985544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:37.985544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:37.985544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:37.986434 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:52:37.986434 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:52:37.986434 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:52:37.986434 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:52:37.986434 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:52:37.986434 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:52:37.986434 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:52:37.986434 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:52:37.986434 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 604/1217] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:52:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:37.991744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16a36b-c93e-4ba6-84dc-d4c688771386 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:38.092280 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16a36b-c93e-4ba6-84dc-d4c688771386 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:38.114247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:38.125334 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e786b906-1729-41ad-93d8-085c7eb0a74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:38.126584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786b906-1729-41ad-93d8-085c7eb0a74e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:38.126973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786b906-1729-41ad-93d8-085c7eb0a74e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:38.127277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786b906-1729-41ad-93d8-085c7eb0a74e None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:38.128051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e786b906-1729-41ad-93d8-085c7eb0a74e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:38.128362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e786b906-1729-41ad-93d8-085c7eb0a74e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:38.129799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e786b906-1729-41ad-93d8-085c7eb0a74e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:38.140817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 614/1218] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:52:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:38.146998 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307a444-c15a-4fbf-aa4b-84bcbac97919 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:38.244931 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307a444-c15a-4fbf-aa4b-84bcbac97919 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:38.272639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:38.278535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e4c59-007e-4c00-918f-29b8c0617914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:38.279197 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e4c59-007e-4c00-918f-29b8c0617914 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:38.279485 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e4c59-007e-4c00-918f-29b8c0617914 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:38.279764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e4c59-007e-4c00-918f-29b8c0617914 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:38.280582 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e4c59-007e-4c00-918f-29b8c0617914 None admin] RBAC: Adding query filter params (role.id=24846e82841b4b05b4d9e9d193a98868, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=c577fa8670e34449a80de0c8e0a15a8b) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:38.280937 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e4c59-007e-4c00-918f-29b8c0617914 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:38.282364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e4c59-007e-4c00-918f-29b8c0617914 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:38.287655 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 605/1219] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:52:38 2022] GET /identity/v3/role_assignments?user.id=c577fa8670e34449a80de0c8e0a15a8b&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=24846e82841b4b05b4d9e9d193a98868 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:40.289935 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a6dad-ac2f-4ee9-99b5-977fc8fb15f4 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:40.290443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a6dad-ac2f-4ee9-99b5-977fc8fb15f4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:40.290443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a6dad-ac2f-4ee9-99b5-977fc8fb15f4 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:40.291864 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 615/1220] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:52:40.297130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349a1e8-6b69-4cef-8f41-001b3bb1c4f4 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:40.297523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349a1e8-6b69-4cef-8f41-001b3bb1c4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:40.297688 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349a1e8-6b69-4cef-8f41-001b3bb1c4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:40.315294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3349a1e8-6b69-4cef-8f41-001b3bb1c4f4 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:52:40.316454 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3349a1e8-6b69-4cef-8f41-001b3bb1c4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:52:40.382187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 606/1221] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:52:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:52:40.404539 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a6dad-ac2f-4ee9-99b5-977fc8fb15f4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:40.507136 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a6dad-ac2f-4ee9-99b5-977fc8fb15f4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:40.540384 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a183e-19ef-41ec-95da-cf811e954375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:40.541112 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a183e-19ef-41ec-95da-cf811e954375 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:40.541397 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a183e-19ef-41ec-95da-cf811e954375 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:40.541672 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a183e-19ef-41ec-95da-cf811e954375 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:40.552602 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a183e-19ef-41ec-95da-cf811e954375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:52:40.560271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-269a183e-19ef-41ec-95da-cf811e954375 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:52:40.560271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:40.560271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:40.560271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:40.560271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:40.560271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:40.560271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:40.560271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:40.560271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:40.560271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:40.560271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:40.560271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:40.560271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:52:40.560271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:52:40.560271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:52:40.560271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:52:40.561376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:52:40.561376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:52:40.561376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:52:40.561376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:52:40.561376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:40.561376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:40.561376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:40.561376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:40.561376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:40.561376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:40.561376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:40.561376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:40.561376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:40.561376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:40.561376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:40.561376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:40.562381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:40.562381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:40.562381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:40.562381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:40.562381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:52:40.562381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:52:40.562381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:52:40.562381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:52:40.562381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:52:40.562381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:52:40.562381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:52:40.562381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:52:40.562381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 616/1222] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:52:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:40.568249 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349a1e8-6b69-4cef-8f41-001b3bb1c4f4 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:40.666121 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349a1e8-6b69-4cef-8f41-001b3bb1c4f4 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:40.690609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:40.700900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c11b9-9342-45df-909e-dcf8bf25c1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:40.701564 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c11b9-9342-45df-909e-dcf8bf25c1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:40.701850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c11b9-9342-45df-909e-dcf8bf25c1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:40.702129 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c11b9-9342-45df-909e-dcf8bf25c1e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:40.702869 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c11b9-9342-45df-909e-dcf8bf25c1e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:40.703203 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c11b9-9342-45df-909e-dcf8bf25c1e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:40.704599 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c11b9-9342-45df-909e-dcf8bf25c1e4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:40.712367 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 607/1223] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:52:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:40.718205 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a183e-19ef-41ec-95da-cf811e954375 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:40.760777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:40.817276 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a183e-19ef-41ec-95da-cf811e954375 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:40.851110 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e774a8f-5bdc-4a0e-9551-67fcb6ef2549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:40.851924 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e774a8f-5bdc-4a0e-9551-67fcb6ef2549 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:40.852210 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e774a8f-5bdc-4a0e-9551-67fcb6ef2549 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:40.852491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e774a8f-5bdc-4a0e-9551-67fcb6ef2549 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:40.862859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e774a8f-5bdc-4a0e-9551-67fcb6ef2549 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:40.864362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e774a8f-5bdc-4a0e-9551-67fcb6ef2549 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:40.881484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-1e774a8f-5bdc-4a0e-9551-67fcb6ef2549 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:40.881484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:40.881484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:40.881484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:40.881484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:40.881484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:40.881484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:40.881484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:40.881484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:40.881484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:40.881484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:40.881484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:40.881484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:52:40.881484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:52:40.881484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:52:40.883006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:52:40.883006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:40.883006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:40.883006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:40.883006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:40.883006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:40.883006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:40.883006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:40.883006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:40.883006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:40.883006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:40.883006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:40.883006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:40.883006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:40.883006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:40.883006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:40.883006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:40.884104 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:52:40.884104 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:52:40.884104 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:40.884104 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:52:40.884104 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 617/1224] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:40.888701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c11b9-9342-45df-909e-dcf8bf25c1e4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:40.987743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c11b9-9342-45df-909e-dcf8bf25c1e4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:41.021889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd6121-2c42-42c4-8707-4a4cf220fe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:41.022541 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd6121-2c42-42c4-8707-4a4cf220fe62 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:41.022825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd6121-2c42-42c4-8707-4a4cf220fe62 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:41.023100 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd6121-2c42-42c4-8707-4a4cf220fe62 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:41.023841 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd6121-2c42-42c4-8707-4a4cf220fe62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:41.024147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd6121-2c42-42c4-8707-4a4cf220fe62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:41.025550 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd6121-2c42-42c4-8707-4a4cf220fe62 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:41.037745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 608/1225] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:41.043468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e774a8f-5bdc-4a0e-9551-67fcb6ef2549 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:41.134987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:41.141331 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e774a8f-5bdc-4a0e-9551-67fcb6ef2549 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:41.173755 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2b668-edde-41e1-9dbc-4273ed8d59ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:41.174547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2b668-edde-41e1-9dbc-4273ed8d59ed None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:41.174828 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2b668-edde-41e1-9dbc-4273ed8d59ed None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:41.175101 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2b668-edde-41e1-9dbc-4273ed8d59ed None admin] PATH_INFO: `/v3/users/placement` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:41.181970 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2b668-edde-41e1-9dbc-4273ed8d59ed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:41.183442 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2b668-edde-41e1-9dbc-4273ed8d59ed None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:41.198226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-f3b2b668-edde-41e1-9dbc-4273ed8d59ed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 18:52:41.198226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:41.198226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:41.198226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:41.198226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:41.198226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:41.198226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:41.198226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:41.198226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:41.198226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:41.198226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:41.198226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:41.198226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:52:41.198226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:52:41.198226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:52:41.198226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:52:41.199221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:41.199221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:41.199221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:52:41.199221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:52:41.199221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:52:41.199221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:52:41.199221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:41.199221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:41.199221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:41.199221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:41.199221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:41.199221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:41.199221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:41.199221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:41.199221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:41.199221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:41.199221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:41.199221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:41.200374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:41.200374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:41.200374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:52:41.200374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:52:41.200374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:52:41.200374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:52:41.200374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:52:41.200374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:52:41.200374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 18:52:41.200374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:52:41.200374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 618/1226] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:41.205311 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd6121-2c42-42c4-8707-4a4cf220fe62 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:41.252257 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:41.303349 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd6121-2c42-42c4-8707-4a4cf220fe62 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:41.336636 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2560129-b0cb-421c-9bda-a05784ac3007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:41.337291 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2560129-b0cb-421c-9bda-a05784ac3007 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:41.337575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2560129-b0cb-421c-9bda-a05784ac3007 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:41.337850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2560129-b0cb-421c-9bda-a05784ac3007 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:41.338710 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2560129-b0cb-421c-9bda-a05784ac3007 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:41.339014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2560129-b0cb-421c-9bda-a05784ac3007 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:41.340460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2560129-b0cb-421c-9bda-a05784ac3007 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:41.357065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 609/1227] 10.222.0.69 () {56 vars in 1143 bytes} [Thu Jul 14 18:52:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:41.363403 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2b668-edde-41e1-9dbc-4273ed8d59ed None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:41.464445 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2b668-edde-41e1-9dbc-4273ed8d59ed None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:41.498542 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2802e69-fc73-401f-b178-7089832a3bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:41.499369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2802e69-fc73-401f-b178-7089832a3bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:41.499659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2802e69-fc73-401f-b178-7089832a3bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:41.499939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2802e69-fc73-401f-b178-7089832a3bd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:41.513436 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2802e69-fc73-401f-b178-7089832a3bd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:41.514915 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2802e69-fc73-401f-b178-7089832a3bd5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:41.534829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c2802e69-fc73-401f-b178-7089832a3bd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:41.534829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:41.534829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:41.534829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:41.534829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:41.534829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:41.534829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:41.534829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:41.534829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:41.534829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:41.534829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:41.534829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:41.534829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:52:41.534829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:52:41.534829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:52:41.535936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:52:41.535936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:41.535936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:41.535936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:41.535936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:41.535936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:41.535936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:41.535936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:41.535936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:41.535936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:41.535936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:41.535936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:41.535936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:41.535936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:41.535936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:41.535936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:41.535936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:41.536965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:52:41.536965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:52:41.536965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:41.536965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:52:41.536965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 619/1228] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:41.541820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2560129-b0cb-421c-9bda-a05784ac3007 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:41.642091 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2560129-b0cb-421c-9bda-a05784ac3007 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:41.644486 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:41.675619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f359c13-85ed-4d6f-9dda-f64904e31092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:41.676285 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f359c13-85ed-4d6f-9dda-f64904e31092 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:41.676572 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f359c13-85ed-4d6f-9dda-f64904e31092 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:41.676851 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f359c13-85ed-4d6f-9dda-f64904e31092 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:41.677572 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f359c13-85ed-4d6f-9dda-f64904e31092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:41.677883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f359c13-85ed-4d6f-9dda-f64904e31092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:41.679269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f359c13-85ed-4d6f-9dda-f64904e31092 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:41.690845 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 610/1229] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:41.696573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2802e69-fc73-401f-b178-7089832a3bd5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:41.706091 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:41.795677 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2802e69-fc73-401f-b178-7089832a3bd5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:41.833017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14891166-3489-4b67-ab95-5aa5e35d7175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:41.833810 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14891166-3489-4b67-ab95-5aa5e35d7175 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:41.834098 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14891166-3489-4b67-ab95-5aa5e35d7175 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:41.834377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14891166-3489-4b67-ab95-5aa5e35d7175 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:41.846830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14891166-3489-4b67-ab95-5aa5e35d7175 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:41.848372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14891166-3489-4b67-ab95-5aa5e35d7175 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:41.862712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-14891166-3489-4b67-ab95-5aa5e35d7175 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:52:41.862712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:41.862712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:41.862712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:41.862712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:41.862712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:41.862712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:41.862712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:41.862712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:41.862712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:41.862712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:41.862712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:41.862712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:52:41.862712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:52:41.862712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:52:41.864295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:52:41.864295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:41.864295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:41.864295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:41.864295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:41.864295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:41.864295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:41.864295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:41.864295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:41.864295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:41.864295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:41.864295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:41.864295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:41.864295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:41.864295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:41.864295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:41.864295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:41.865422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:52:41.865422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:52:41.865422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:52:41.865422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:52:41.865422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:52:41.865422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:52:41.865422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:52:41.865422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:52:41.865422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 620/1230] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:52:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:41.869824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f359c13-85ed-4d6f-9dda-f64904e31092 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:41.974817 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f359c13-85ed-4d6f-9dda-f64904e31092 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:42.008013 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c49f8c-a779-404e-8687-baaf3593b696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:42.008693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c49f8c-a779-404e-8687-baaf3593b696 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:42.008981 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c49f8c-a779-404e-8687-baaf3593b696 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:42.009262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c49f8c-a779-404e-8687-baaf3593b696 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:42.010044 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c49f8c-a779-404e-8687-baaf3593b696 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:42.010355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c49f8c-a779-404e-8687-baaf3593b696 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:42.011889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c49f8c-a779-404e-8687-baaf3593b696 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:42.024499 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 611/1231] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:52:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:42.030902 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14891166-3489-4b67-ab95-5aa5e35d7175 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:42.096893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:42.131332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14891166-3489-4b67-ab95-5aa5e35d7175 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:42.164096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fa00c-1617-4ef6-924d-d0563ea9150b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:42.164985 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fa00c-1617-4ef6-924d-d0563ea9150b None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:42.165273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fa00c-1617-4ef6-924d-d0563ea9150b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:42.165552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fa00c-1617-4ef6-924d-d0563ea9150b None admin] PATH_INFO: `/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/c577fa8670e34449a80de0c8e0a15a8b/roles/24846e82841b4b05b4d9e9d193a98868` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:42.181390 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5fa00c-1617-4ef6-924d-d0563ea9150b None admin] RBAC: Authorizing `identity:create_grant(project_id=6003dba8ddd247e7842d28ea7ca5238c, user_id=c577fa8670e34449a80de0c8e0a15a8b, role_id=24846e82841b4b05b4d9e9d193a98868)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:42.182913 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5fa00c-1617-4ef6-924d-d0563ea9150b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:42.251538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 621/1232] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:52:42 2022] PUT /identity/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/c577fa8670e34449a80de0c8e0a15a8b/roles/24846e82841b4b05b4d9e9d193a98868 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:52:44.230442 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb1a52-5d62-4cb7-9fff-d4a5cda64c8e None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:44.230961 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb1a52-5d62-4cb7-9fff-d4a5cda64c8e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:44.231049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb1a52-5d62-4cb7-9fff-d4a5cda64c8e None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:44.232407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 612/1233] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:52:44.238057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c35db8-c643-4c0e-aafe-9743c1f4b41f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:44.238433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c35db8-c643-4c0e-aafe-9743c1f4b41f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:44.238632 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c35db8-c643-4c0e-aafe-9743c1f4b41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:44.269542 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-17c35db8-c643-4c0e-aafe-9743c1f4b41f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:52:44.269542 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-17c35db8-c643-4c0e-aafe-9743c1f4b41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:52:44.332589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 622/1234] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:52:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:52:44.355227 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb1a52-5d62-4cb7-9fff-d4a5cda64c8e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:44.381017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:44.457044 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb1a52-5d62-4cb7-9fff-d4a5cda64c8e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:44.490359 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3de672-4a60-4952-b031-dee0f23f3983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:44.491203 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3de672-4a60-4952-b031-dee0f23f3983 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:44.491490 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3de672-4a60-4952-b031-dee0f23f3983 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:44.491763 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3de672-4a60-4952-b031-dee0f23f3983 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:44.502233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3de672-4a60-4952-b031-dee0f23f3983 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:52:44.510039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-6e3de672-4a60-4952-b031-dee0f23f3983 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:52:44.510039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:44.510039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:44.510039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:44.510039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:44.510039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:44.510039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:44.510039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:44.510039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:44.510039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:44.510039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:44.510039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:44.510039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:52:44.510039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:52:44.510039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:52:44.510039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:52:44.511159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:52:44.511159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:52:44.511159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:52:44.511159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:52:44.511159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:44.511159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:44.511159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:44.511159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:44.511159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:44.511159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:44.511159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:44.511159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:44.511159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:44.511159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:44.511159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:44.511159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:44.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:44.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:44.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:44.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:44.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:52:44.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:52:44.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:52:44.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:52:44.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:52:44.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:52:44.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 18:52:44.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:52:44.512207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 613/1235] 10.222.0.69 () {56 vars in 1102 bytes} [Thu Jul 14 18:52:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:44.518120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c35db8-c643-4c0e-aafe-9743c1f4b41f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:44.563851 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:44.617393 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c35db8-c643-4c0e-aafe-9743c1f4b41f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:44.651399 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9af73-b705-4b2d-af0b-757729715634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:44.652066 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9af73-b705-4b2d-af0b-757729715634 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:44.652352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9af73-b705-4b2d-af0b-757729715634 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:44.652697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9af73-b705-4b2d-af0b-757729715634 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:44.653650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9af73-b705-4b2d-af0b-757729715634 None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:44.653975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9af73-b705-4b2d-af0b-757729715634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:44.655363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9af73-b705-4b2d-af0b-757729715634 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:44.663327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 623/1236] 10.222.0.69 () {56 vars in 1103 bytes} [Thu Jul 14 18:52:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:44.669194 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3de672-4a60-4952-b031-dee0f23f3983 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:44.752591 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:44.768217 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3de672-4a60-4952-b031-dee0f23f3983 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:44.802476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeab8cc-7030-4686-904f-71830f2cbf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:44.803299 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeab8cc-7030-4686-904f-71830f2cbf9b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:44.803586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeab8cc-7030-4686-904f-71830f2cbf9b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:44.803880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeab8cc-7030-4686-904f-71830f2cbf9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:44.813739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeab8cc-7030-4686-904f-71830f2cbf9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:44.817902 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeab8cc-7030-4686-904f-71830f2cbf9b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:44.832747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-faeab8cc-7030-4686-904f-71830f2cbf9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:44.832747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:44.832747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:44.832747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:44.832747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:44.832747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:44.832747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:44.832747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:44.832747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:44.832747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:44.832747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:44.832747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:44.832747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:52:44.832747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:52:44.832747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:52:44.833608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:52:44.833608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:44.833608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:44.833608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:44.833608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:44.833608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:44.833608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:44.833608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:44.833608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:44.833608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:44.833608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:44.833608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:44.833608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:44.833608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:44.833608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:44.833608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:44.833608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:44.834491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:52:44.834491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:52:44.834491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:44.834491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:52:44.834491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 614/1237] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:44.839800 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9af73-b705-4b2d-af0b-757729715634 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:44.942571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9af73-b705-4b2d-af0b-757729715634 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:44.951746 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:44.976819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1170a1-e3cd-4788-9441-01aa97acf5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:44.977485 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1170a1-e3cd-4788-9441-01aa97acf5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:44.977776 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1170a1-e3cd-4788-9441-01aa97acf5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:44.978055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1170a1-e3cd-4788-9441-01aa97acf5d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:44.978775 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1170a1-e3cd-4788-9441-01aa97acf5d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:44.979088 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1170a1-e3cd-4788-9441-01aa97acf5d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:44.980508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1170a1-e3cd-4788-9441-01aa97acf5d5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:44.992725 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 624/1238] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:44.998612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeab8cc-7030-4686-904f-71830f2cbf9b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:45.097540 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeab8cc-7030-4686-904f-71830f2cbf9b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:45.132242 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5151503-bc9a-4b21-9309-5389b728aaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:45.133045 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5151503-bc9a-4b21-9309-5389b728aaa1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:45.133335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5151503-bc9a-4b21-9309-5389b728aaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:45.133622 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5151503-bc9a-4b21-9309-5389b728aaa1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:45.136380 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:45.140946 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5151503-bc9a-4b21-9309-5389b728aaa1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:45.142389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5151503-bc9a-4b21-9309-5389b728aaa1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:45.157113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-f5151503-bc9a-4b21-9309-5389b728aaa1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 18:52:45.157113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:45.157113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:45.157113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:45.157113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:45.157113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:45.157113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:45.157113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:45.157113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:45.157113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:45.157113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:45.157113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:45.157113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:52:45.157113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:52:45.157113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:52:45.157113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:52:45.158161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:45.158161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:45.158161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:52:45.158161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:52:45.158161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:52:45.158161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:52:45.158161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:45.158161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:45.158161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:45.158161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:45.158161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:45.158161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:45.158161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:45.158161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:45.158161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:45.158161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:45.158161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:45.158161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:45.169674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:45.169674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:45.169674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:52:45.169674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:52:45.169674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:52:45.169674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:52:45.169674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:52:45.169674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:52:45.169674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 18:52:45.169674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:52:45.169674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 615/1239] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:45.170389 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1170a1-e3cd-4788-9441-01aa97acf5d5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:45.269569 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1170a1-e3cd-4788-9441-01aa97acf5d5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:45.303608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d6f66-b793-4eb6-9656-5234e1aee270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:45.304277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d6f66-b793-4eb6-9656-5234e1aee270 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:45.304561 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d6f66-b793-4eb6-9656-5234e1aee270 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:45.304831 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d6f66-b793-4eb6-9656-5234e1aee270 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:45.305645 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d6f66-b793-4eb6-9656-5234e1aee270 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:45.305951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d6f66-b793-4eb6-9656-5234e1aee270 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:45.307630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d6f66-b793-4eb6-9656-5234e1aee270 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:45.324474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 625/1240] 10.222.0.69 () {56 vars in 1143 bytes} [Thu Jul 14 18:52:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:45.330229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5151503-bc9a-4b21-9309-5389b728aaa1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:45.430494 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5151503-bc9a-4b21-9309-5389b728aaa1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:45.463377 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae8d56-a955-4d48-828e-9ad0580403b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:45.464020 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae8d56-a955-4d48-828e-9ad0580403b0 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:45.464304 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae8d56-a955-4d48-828e-9ad0580403b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:45.464690 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae8d56-a955-4d48-828e-9ad0580403b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:45.546028 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:45.552747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae8d56-a955-4d48-828e-9ad0580403b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:45.554320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae8d56-a955-4d48-828e-9ad0580403b0 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:45.651264 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 616/1241] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:52:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:52:45.657523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d6f66-b793-4eb6-9656-5234e1aee270 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:45.685435 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:45.756257 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d6f66-b793-4eb6-9656-5234e1aee270 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:45.789083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840c3c8-708d-45c8-a868-3efa9ecaf6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:45.789861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840c3c8-708d-45c8-a868-3efa9ecaf6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:45.790147 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840c3c8-708d-45c8-a868-3efa9ecaf6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:45.790428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840c3c8-708d-45c8-a868-3efa9ecaf6a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:45.800778 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840c3c8-708d-45c8-a868-3efa9ecaf6a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:45.802222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840c3c8-708d-45c8-a868-3efa9ecaf6a5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:45.819875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c840c3c8-708d-45c8-a868-3efa9ecaf6a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:45.819875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:45.819875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:45.819875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:45.819875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:45.819875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:45.819875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:45.819875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:45.819875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:45.819875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:45.819875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:45.819875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:45.819875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:52:45.819875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:52:45.819875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:52:45.820919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:52:45.820919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:45.820919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:45.820919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:45.820919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:45.820919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:45.820919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:45.820919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:45.820919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:45.820919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:45.820919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:45.820919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:45.820919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:45.820919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:45.820919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:45.820919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:45.820919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:45.822015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:52:45.822015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:52:45.822015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:45.822015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:52:45.822015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 626/1242] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:45.826762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae8d56-a955-4d48-828e-9ad0580403b0 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:45.920025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:45.926301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae8d56-a955-4d48-828e-9ad0580403b0 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:45.961018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c588c4-62c5-4d96-a901-4ef2af420bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:45.961617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c588c4-62c5-4d96-a901-4ef2af420bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:45.961906 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c588c4-62c5-4d96-a901-4ef2af420bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:45.962184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c588c4-62c5-4d96-a901-4ef2af420bf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:45.962903 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c588c4-62c5-4d96-a901-4ef2af420bf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:45.963240 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c588c4-62c5-4d96-a901-4ef2af420bf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:45.964621 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c588c4-62c5-4d96-a901-4ef2af420bf5 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:45.976144 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 617/1243] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:52:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:45.981916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840c3c8-708d-45c8-a868-3efa9ecaf6a5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:46.061098 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:46.081560 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840c3c8-708d-45c8-a868-3efa9ecaf6a5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:46.114854 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a2635-8c41-43ac-bb87-c71da017538a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:46.115672 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a2635-8c41-43ac-bb87-c71da017538a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:46.115965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a2635-8c41-43ac-bb87-c71da017538a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:46.116248 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a2635-8c41-43ac-bb87-c71da017538a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:46.122842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a2635-8c41-43ac-bb87-c71da017538a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:46.124326 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a2635-8c41-43ac-bb87-c71da017538a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:46.137806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-a90a2635-8c41-43ac-bb87-c71da017538a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:52:46.137806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:46.137806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:46.137806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:46.137806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:46.137806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:46.137806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:46.137806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:46.137806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:46.137806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:46.137806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:46.137806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:46.137806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:52:46.137806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:52:46.137806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:52:46.138662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:52:46.138662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:46.138662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:46.138662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:46.138662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:46.138662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:46.138662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:46.138662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:46.138662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:46.138662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:46.138662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:46.138662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:46.138662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:46.138662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:46.138662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:46.138662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:46.138662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:46.139584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:52:46.139584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:52:46.139584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:52:46.139584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:52:46.139584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:52:46.139584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:52:46.139584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:52:46.139584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:52:46.139584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 627/1244] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:52:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:46.144946 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c588c4-62c5-4d96-a901-4ef2af420bf5 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:46.241783 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c588c4-62c5-4d96-a901-4ef2af420bf5 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:46.274344 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d989e-89d8-47dc-8bfc-c330c4ccabf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:46.275000 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d989e-89d8-47dc-8bfc-c330c4ccabf3 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:46.275406 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d989e-89d8-47dc-8bfc-c330c4ccabf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:46.281095 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d989e-89d8-47dc-8bfc-c330c4ccabf3 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:46.281095 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d989e-89d8-47dc-8bfc-c330c4ccabf3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:46.281095 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d989e-89d8-47dc-8bfc-c330c4ccabf3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:46.281095 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d989e-89d8-47dc-8bfc-c330c4ccabf3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:46.291445 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 618/1245] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:52:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:46.297538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a2635-8c41-43ac-bb87-c71da017538a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:46.395752 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a2635-8c41-43ac-bb87-c71da017538a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:46.429421 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d502c-4fdd-41de-81cb-0cead92da80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:46.430078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d502c-4fdd-41de-81cb-0cead92da80d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:46.430371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d502c-4fdd-41de-81cb-0cead92da80d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:46.430651 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d502c-4fdd-41de-81cb-0cead92da80d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:46.431509 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d502c-4fdd-41de-81cb-0cead92da80d None admin] RBAC: Adding query filter params (role.id=24846e82841b4b05b4d9e9d193a98868, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=c577fa8670e34449a80de0c8e0a15a8b) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:46.431825 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d502c-4fdd-41de-81cb-0cead92da80d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:46.433275 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d502c-4fdd-41de-81cb-0cead92da80d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:46.439150 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 628/1246] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:52:46 2022] GET /identity/v3/role_assignments?user.id=c577fa8670e34449a80de0c8e0a15a8b&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=24846e82841b4b05b4d9e9d193a98868 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:48.455810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfd896-b6ee-4a61-a1c1-e682e32170a1 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:48.456381 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfd896-b6ee-4a61-a1c1-e682e32170a1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:48.456381 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfd896-b6ee-4a61-a1c1-e682e32170a1 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:48.457807 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 619/1247] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:52:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:52:48.462851 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a95430-d3bb-44e8-babc-d93aeee39b1c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:48.463294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a95430-d3bb-44e8-babc-d93aeee39b1c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:48.463452 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a95430-d3bb-44e8-babc-d93aeee39b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:48.468589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:48.480561 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d7a95430-d3bb-44e8-babc-d93aeee39b1c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:52:48.481739 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d7a95430-d3bb-44e8-babc-d93aeee39b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:52:48.546218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 629/1248] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:52:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:52:48.568960 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfd896-b6ee-4a61-a1c1-e682e32170a1 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:48.588297 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:48.669486 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfd896-b6ee-4a61-a1c1-e682e32170a1 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:48.702997 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42f161-bca7-47a7-be70-32eca9a50faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:48.704331 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42f161-bca7-47a7-be70-32eca9a50faa None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:48.704617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42f161-bca7-47a7-be70-32eca9a50faa None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:48.704893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42f161-bca7-47a7-be70-32eca9a50faa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:48.714758 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d42f161-bca7-47a7-be70-32eca9a50faa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:52:48.722930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-5d42f161-bca7-47a7-be70-32eca9a50faa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:52:48.722930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:48.722930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:48.722930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:48.722930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:48.722930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:48.722930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:48.722930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:48.722930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:48.722930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:48.722930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:48.722930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:48.722930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:52:48.722930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:52:48.722930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:52:48.722930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:52:48.726039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:52:48.726039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:52:48.726039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:52:48.726039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:52:48.726039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:48.726039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:48.726039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:48.726039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:48.726039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:48.726039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:48.726039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:48.726039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:48.726039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:48.726039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:48.726039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:48.726039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:48.727060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:48.727060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:48.727060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:48.727060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:48.727060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:52:48.727060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:52:48.727060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:52:48.727060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:52:48.727060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:52:48.727060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:52:48.727060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:52:48.727060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:52:48.727060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 620/1249] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:52:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:48.730254 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a95430-d3bb-44e8-babc-d93aeee39b1c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:48.826319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a95430-d3bb-44e8-babc-d93aeee39b1c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:48.856558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:48.861580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401d4d3-80e6-456d-ae9b-5944ce9b138c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:48.862250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401d4d3-80e6-456d-ae9b-5944ce9b138c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:48.862541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401d4d3-80e6-456d-ae9b-5944ce9b138c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:48.862823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401d4d3-80e6-456d-ae9b-5944ce9b138c None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:48.863638 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f401d4d3-80e6-456d-ae9b-5944ce9b138c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:48.863951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f401d4d3-80e6-456d-ae9b-5944ce9b138c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:48.865345 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f401d4d3-80e6-456d-ae9b-5944ce9b138c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:48.873157 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 630/1250] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:52:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:48.879028 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42f161-bca7-47a7-be70-32eca9a50faa None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:48.956820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:48.979271 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42f161-bca7-47a7-be70-32eca9a50faa None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:49.011585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b21ca-3f6e-4853-99db-c867d47e0c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:49.012415 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b21ca-3f6e-4853-99db-c867d47e0c86 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:49.012697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b21ca-3f6e-4853-99db-c867d47e0c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:49.013437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b21ca-3f6e-4853-99db-c867d47e0c86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:49.025408 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b21ca-3f6e-4853-99db-c867d47e0c86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:49.026856 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b21ca-3f6e-4853-99db-c867d47e0c86 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:49.044895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-c84b21ca-3f6e-4853-99db-c867d47e0c86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:49.044895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:49.044895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:49.044895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:49.044895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:49.044895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:49.044895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:49.044895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:49.044895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:49.044895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:49.044895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:49.044895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:49.044895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:52:49.044895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:52:49.044895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:52:49.045935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:52:49.045935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:49.045935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:49.045935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:49.045935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:49.045935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:49.045935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:49.045935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:49.045935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:49.045935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:49.045935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:49.045935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:49.045935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:49.045935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:49.045935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:49.045935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:49.045935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:49.046910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:52:49.046910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:52:49.046910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:49.046910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:52:49.046910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 621/1251] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:49.051977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401d4d3-80e6-456d-ae9b-5944ce9b138c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:49.150648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401d4d3-80e6-456d-ae9b-5944ce9b138c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:49.183103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed8113-b16f-4490-b1df-9322e57806a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:49.183817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed8113-b16f-4490-b1df-9322e57806a4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:49.184103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed8113-b16f-4490-b1df-9322e57806a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:49.184380 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed8113-b16f-4490-b1df-9322e57806a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:49.185094 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed8113-b16f-4490-b1df-9322e57806a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:49.185401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed8113-b16f-4490-b1df-9322e57806a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:49.186776 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed8113-b16f-4490-b1df-9322e57806a4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:49.199059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 631/1252] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:49.205480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b21ca-3f6e-4853-99db-c867d47e0c86 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:49.304429 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b21ca-3f6e-4853-99db-c867d47e0c86 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:49.330428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:49.337877 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56ddcd-e091-422b-b4d4-5552ab8343c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:49.341287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56ddcd-e091-422b-b4d4-5552ab8343c6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:49.341287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56ddcd-e091-422b-b4d4-5552ab8343c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:49.341287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56ddcd-e091-422b-b4d4-5552ab8343c6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:49.346488 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56ddcd-e091-422b-b4d4-5552ab8343c6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:49.347995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56ddcd-e091-422b-b4d4-5552ab8343c6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:49.362365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-df56ddcd-e091-422b-b4d4-5552ab8343c6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 18:52:49.362365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:49.362365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:49.362365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:49.362365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:49.362365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:49.362365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:49.362365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:49.362365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:49.362365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:49.362365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:49.362365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:49.362365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:52:49.362365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:52:49.362365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:52:49.362365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:52:49.363509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:49.363509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:49.363509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:52:49.363509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:52:49.363509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:52:49.363509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:52:49.363509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:49.363509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:49.363509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:49.363509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:49.363509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:49.363509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:49.363509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:49.363509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:49.363509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:49.363509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:49.363509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:49.363509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:49.364527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:49.364527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:49.364527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:52:49.364527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:52:49.364527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:52:49.364527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:52:49.364527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:52:49.364527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:52:49.364527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 18:52:49.364527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:52:49.364527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 622/1253] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:49.369667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed8113-b16f-4490-b1df-9322e57806a4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:49.425158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:49.468860 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed8113-b16f-4490-b1df-9322e57806a4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:49.502511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c9195-e8f7-48c3-bd5a-2ad8a787b577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:49.503203 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c9195-e8f7-48c3-bd5a-2ad8a787b577 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:49.503496 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c9195-e8f7-48c3-bd5a-2ad8a787b577 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:49.503775 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c9195-e8f7-48c3-bd5a-2ad8a787b577 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:49.504602 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c9195-e8f7-48c3-bd5a-2ad8a787b577 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:49.504978 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c9195-e8f7-48c3-bd5a-2ad8a787b577 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:49.506417 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c9195-e8f7-48c3-bd5a-2ad8a787b577 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:49.524136 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 632/1254] 10.222.0.69 () {56 vars in 1143 bytes} [Thu Jul 14 18:52:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:49.530174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56ddcd-e091-422b-b4d4-5552ab8343c6 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:49.630018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56ddcd-e091-422b-b4d4-5552ab8343c6 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:49.663762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b74ca-4206-4c3a-b374-5740ca492097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:49.664451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b74ca-4206-4c3a-b374-5740ca492097 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:49.664735 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b74ca-4206-4c3a-b374-5740ca492097 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:49.665010 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b74ca-4206-4c3a-b374-5740ca492097 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:49.718999 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:49.731183 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b74ca-4206-4c3a-b374-5740ca492097 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:49.732705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b74ca-4206-4c3a-b374-5740ca492097 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:49.829130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 623/1255] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:52:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:52:49.834903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c9195-e8f7-48c3-bd5a-2ad8a787b577 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:49.931653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c9195-e8f7-48c3-bd5a-2ad8a787b577 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:49.944781 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:49.964946 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d67b2b-6660-4aa3-8ba0-4750cbeb596b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:49.965736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d67b2b-6660-4aa3-8ba0-4750cbeb596b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:49.966040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d67b2b-6660-4aa3-8ba0-4750cbeb596b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:49.966481 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d67b2b-6660-4aa3-8ba0-4750cbeb596b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:49.976959 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d67b2b-6660-4aa3-8ba0-4750cbeb596b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:49.978472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d67b2b-6660-4aa3-8ba0-4750cbeb596b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:49.996056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-d0d67b2b-6660-4aa3-8ba0-4750cbeb596b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:49.996056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:49.996056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:49.996056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:49.996056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:49.996056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:49.996056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:49.996056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:49.996056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:49.996056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:49.996056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:49.996056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:49.996056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:52:49.996056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:52:49.996056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:52:49.997102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:52:49.997102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:49.997102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:49.997102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:49.997102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:49.997102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:49.997102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:49.997102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:49.997102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:49.997102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:49.997102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:49.997102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:49.997102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:49.997102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:49.997102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:49.997102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:49.997102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:49.998030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:52:49.998030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:52:49.998030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:49.998030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:52:49.998030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 633/1256] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:50.003190 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b74ca-4206-4c3a-b374-5740ca492097 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:50.094836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:50.106314 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b74ca-4206-4c3a-b374-5740ca492097 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:50.139102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445c2ae-dc94-45dc-a9b9-5f753efac51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:50.139795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445c2ae-dc94-45dc-a9b9-5f753efac51f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:50.140081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445c2ae-dc94-45dc-a9b9-5f753efac51f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:50.140359 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445c2ae-dc94-45dc-a9b9-5f753efac51f None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:50.141602 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445c2ae-dc94-45dc-a9b9-5f753efac51f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:50.141913 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445c2ae-dc94-45dc-a9b9-5f753efac51f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:50.143309 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445c2ae-dc94-45dc-a9b9-5f753efac51f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:50.154679 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 624/1257] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:50.161199 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d67b2b-6660-4aa3-8ba0-4750cbeb596b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:50.261475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d67b2b-6660-4aa3-8ba0-4750cbeb596b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:50.296463 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8e1e1-d08f-4699-8c61-244116903b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:50.297277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8e1e1-d08f-4699-8c61-244116903b1d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:50.297563 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8e1e1-d08f-4699-8c61-244116903b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:50.297838 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8e1e1-d08f-4699-8c61-244116903b1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:50.304618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8e1e1-d08f-4699-8c61-244116903b1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:50.306078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8e1e1-d08f-4699-8c61-244116903b1d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:50.319878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c8a8e1e1-d08f-4699-8c61-244116903b1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:52:50.319878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:50.319878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:50.319878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:50.319878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:50.319878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:50.319878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:50.319878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:50.319878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:50.319878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:50.319878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:50.319878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:50.319878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:52:50.319878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:52:50.319878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:52:50.327990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:52:50.327990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:50.327990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:50.327990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:50.327990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:50.327990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:50.327990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:50.327990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:50.327990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:50.327990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:50.327990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:50.327990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:50.327990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:50.327990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:50.327990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:50.327990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:50.327990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:50.329668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:52:50.329668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:52:50.329668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:52:50.329668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:52:50.329668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:52:50.329668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:52:50.329668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:52:50.329668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:52:50.329668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 634/1258] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:52:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:50.330299 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445c2ae-dc94-45dc-a9b9-5f753efac51f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:50.429537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445c2ae-dc94-45dc-a9b9-5f753efac51f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:50.462397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52961b19-f5d9-4fae-bb64-467f2e7b7ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:50.463042 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52961b19-f5d9-4fae-bb64-467f2e7b7ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:50.463355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52961b19-f5d9-4fae-bb64-467f2e7b7ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:50.463631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52961b19-f5d9-4fae-bb64-467f2e7b7ca3 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:50.464397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52961b19-f5d9-4fae-bb64-467f2e7b7ca3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:50.464705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52961b19-f5d9-4fae-bb64-467f2e7b7ca3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:50.466252 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52961b19-f5d9-4fae-bb64-467f2e7b7ca3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:50.479182 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 625/1259] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:52:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:50.485639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8e1e1-d08f-4699-8c61-244116903b1d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:50.512271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:50.587413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8e1e1-d08f-4699-8c61-244116903b1d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:50.619649 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ae25f-4049-470b-a7b4-ab16f89aefa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:50.620336 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ae25f-4049-470b-a7b4-ab16f89aefa3 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:50.620618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ae25f-4049-470b-a7b4-ab16f89aefa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:50.620893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ae25f-4049-470b-a7b4-ab16f89aefa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:50.621712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ae25f-4049-470b-a7b4-ab16f89aefa3 None admin] RBAC: Adding query filter params (role.id=27fc14631c494ca0b2dc7f708be016c2, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=c577fa8670e34449a80de0c8e0a15a8b) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:50.622021 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ae25f-4049-470b-a7b4-ab16f89aefa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:50.623799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ae25f-4049-470b-a7b4-ab16f89aefa3 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:50.630295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 635/1260] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:52:50 2022] GET /identity/v3/role_assignments?user.id=c577fa8670e34449a80de0c8e0a15a8b&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:52.600468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782ce32-5d1f-4d6a-aba5-070cde338dad None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:52.601057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782ce32-5d1f-4d6a-aba5-070cde338dad None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:52.601057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782ce32-5d1f-4d6a-aba5-070cde338dad None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:52.602411 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 626/1261] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:52:52.607520 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f1ff2-df61-42e3-afb9-fbb8d2c255aa None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:52.607964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f1ff2-df61-42e3-afb9-fbb8d2c255aa None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:52.608064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f1ff2-df61-42e3-afb9-fbb8d2c255aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:52.624984 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-a50f1ff2-df61-42e3-afb9-fbb8d2c255aa None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:52:52.626739 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a50f1ff2-df61-42e3-afb9-fbb8d2c255aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:52:52.692730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 636/1262] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:52:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:52:52.714320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c782ce32-5d1f-4d6a-aba5-070cde338dad None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:52.725832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:52.815294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c782ce32-5d1f-4d6a-aba5-070cde338dad None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:52.848488 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d63b5b-e799-43b8-834a-b6ef744590df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:52.849249 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d63b5b-e799-43b8-834a-b6ef744590df None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:52.849539 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d63b5b-e799-43b8-834a-b6ef744590df None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:52.849821 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d63b5b-e799-43b8-834a-b6ef744590df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:52.866676 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d63b5b-e799-43b8-834a-b6ef744590df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:52:52.869237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-82d63b5b-e799-43b8-834a-b6ef744590df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:52:52.869237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:52.869237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:52.869237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:52.869237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:52.869237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:52.869237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:52.869237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:52.869237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:52.869237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:52.869237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:52.869237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:52.869237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:52:52.869237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:52:52.869237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:52:52.869237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:52:52.870384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:52:52.870384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:52:52.870384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:52:52.870384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:52:52.870384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:52.870384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:52.870384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:52.870384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:52.870384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:52.870384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:52.870384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:52.870384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:52.870384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:52.870384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:52.870384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:52.870384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:52.871525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:52.871525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:52.871525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:52.871525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:52.871525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:52:52.871525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:52:52.871525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:52:52.871525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:52:52.871525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:52:52.871525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:52:52.871525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:52:52.871525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:52:52.871525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 627/1263] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:52:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:52.876813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f1ff2-df61-42e3-afb9-fbb8d2c255aa None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:52.894977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:52.973445 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f1ff2-df61-42e3-afb9-fbb8d2c255aa None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:53.009534 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97389f-5f66-4045-9cc7-68dcb00f0cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:53.010217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97389f-5f66-4045-9cc7-68dcb00f0cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:53.010506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97389f-5f66-4045-9cc7-68dcb00f0cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:53.010786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97389f-5f66-4045-9cc7-68dcb00f0cc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:53.011568 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97389f-5f66-4045-9cc7-68dcb00f0cc6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:53.011880 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97389f-5f66-4045-9cc7-68dcb00f0cc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:53.013255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97389f-5f66-4045-9cc7-68dcb00f0cc6 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:53.021348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 637/1264] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:52:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:53.027007 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d63b5b-e799-43b8-834a-b6ef744590df None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:53.096322 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:53.128412 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d63b5b-e799-43b8-834a-b6ef744590df None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:53.161727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748eabb5-62d9-4bcf-9998-66e8a0b5eb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:53.162558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748eabb5-62d9-4bcf-9998-66e8a0b5eb99 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:53.163285 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748eabb5-62d9-4bcf-9998-66e8a0b5eb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:53.168123 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748eabb5-62d9-4bcf-9998-66e8a0b5eb99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:53.173899 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748eabb5-62d9-4bcf-9998-66e8a0b5eb99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:53.175370 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748eabb5-62d9-4bcf-9998-66e8a0b5eb99 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:53.192228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-748eabb5-62d9-4bcf-9998-66e8a0b5eb99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:53.192228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:53.192228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:53.192228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:53.192228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:53.192228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:53.192228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:53.192228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:53.192228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:53.192228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:53.192228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:53.192228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:53.192228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:52:53.192228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:52:53.192228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:52:53.197900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:52:53.197900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:53.197900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:53.197900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:53.197900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:53.197900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:53.197900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:53.197900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:53.197900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:53.197900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:53.197900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:53.197900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:53.197900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:53.197900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:53.197900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:53.197900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:53.197900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:53.198886 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:52:53.198886 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:52:53.198886 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:53.198886 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:52:53.198886 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 628/1265] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:53.199496 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97389f-5f66-4045-9cc7-68dcb00f0cc6 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:53.278958 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:53.297902 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97389f-5f66-4045-9cc7-68dcb00f0cc6 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:53.331491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29336a92-2e21-4ab2-bcbd-2945960d28c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:53.332145 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29336a92-2e21-4ab2-bcbd-2945960d28c4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:53.332432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29336a92-2e21-4ab2-bcbd-2945960d28c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:53.332710 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29336a92-2e21-4ab2-bcbd-2945960d28c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:53.333417 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29336a92-2e21-4ab2-bcbd-2945960d28c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:53.333723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29336a92-2e21-4ab2-bcbd-2945960d28c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:53.335096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29336a92-2e21-4ab2-bcbd-2945960d28c4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:53.347406 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 638/1266] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:53.356235 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748eabb5-62d9-4bcf-9998-66e8a0b5eb99 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:53.464585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748eabb5-62d9-4bcf-9998-66e8a0b5eb99 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:53.483904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:53.497386 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045b983-f05c-410b-ba16-c0c3cd6e6077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:53.498181 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045b983-f05c-410b-ba16-c0c3cd6e6077 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:53.498471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045b983-f05c-410b-ba16-c0c3cd6e6077 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:53.498751 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045b983-f05c-410b-ba16-c0c3cd6e6077 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:53.506578 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6045b983-f05c-410b-ba16-c0c3cd6e6077 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:53.508041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6045b983-f05c-410b-ba16-c0c3cd6e6077 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:53.522823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-6045b983-f05c-410b-ba16-c0c3cd6e6077 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 18:52:53.522823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:53.522823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:53.522823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:53.522823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:53.522823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:53.522823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:53.522823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:53.522823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:53.522823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:53.522823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:53.522823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:53.522823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:52:53.522823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:52:53.522823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:52:53.522823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:52:53.524034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:53.524034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:53.524034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:52:53.524034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:52:53.524034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:52:53.524034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:52:53.524034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:53.524034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:53.524034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:53.524034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:53.524034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:53.524034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:53.524034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:53.524034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:53.524034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:53.524034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:53.524034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:53.524034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:53.525571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:53.525571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:53.525571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:52:53.525571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:52:53.525571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:52:53.525571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:52:53.525571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:52:53.525571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:52:53.525571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 18:52:53.525571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:52:53.525571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 629/1267] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:53.530119 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29336a92-2e21-4ab2-bcbd-2945960d28c4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:53.630319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29336a92-2e21-4ab2-bcbd-2945960d28c4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:53.663308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b84cdf-a4d0-4360-b659-d71b308b7059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:53.664018 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b84cdf-a4d0-4360-b659-d71b308b7059 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:53.664302 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b84cdf-a4d0-4360-b659-d71b308b7059 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:53.664573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b84cdf-a4d0-4360-b659-d71b308b7059 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:53.665389 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b84cdf-a4d0-4360-b659-d71b308b7059 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:53.665694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b84cdf-a4d0-4360-b659-d71b308b7059 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:53.667142 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b84cdf-a4d0-4360-b659-d71b308b7059 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:53.676154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:53.684396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 639/1268] 10.222.0.69 () {56 vars in 1143 bytes} [Thu Jul 14 18:52:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:53.690624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045b983-f05c-410b-ba16-c0c3cd6e6077 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:53.792092 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045b983-f05c-410b-ba16-c0c3cd6e6077 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:53.825547 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92da1d-41d1-4a42-aaf0-ac572825c87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:53.826283 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92da1d-41d1-4a42-aaf0-ac572825c87a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:53.826568 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92da1d-41d1-4a42-aaf0-ac572825c87a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:53.826845 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92da1d-41d1-4a42-aaf0-ac572825c87a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:53.837772 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92da1d-41d1-4a42-aaf0-ac572825c87a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:53.840439 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92da1d-41d1-4a42-aaf0-ac572825c87a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:53.857112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-ed92da1d-41d1-4a42-aaf0-ac572825c87a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:53.857112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:53.857112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:53.857112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:53.857112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:53.857112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:53.857112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:53.857112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:53.857112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:53.857112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:53.857112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:53.857112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:53.857112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:52:53.857112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:52:53.857112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:52:53.858187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:52:53.858187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:53.858187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:53.858187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:53.858187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:53.858187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:53.858187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:53.858187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:53.858187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:53.858187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:53.858187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:53.858187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:53.858187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:53.858187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:53.858187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:53.858187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:53.858187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:53.859281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:52:53.859281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:52:53.859281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:53.859281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:52:53.859281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 630/1269] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:53.864156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b84cdf-a4d0-4360-b659-d71b308b7059 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:53.962351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b84cdf-a4d0-4360-b659-d71b308b7059 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:53.996576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1a431-e2b8-4a49-966f-749fcbefbac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:53.997279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1a431-e2b8-4a49-966f-749fcbefbac4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:53.997831 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1a431-e2b8-4a49-966f-749fcbefbac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:53.998380 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1a431-e2b8-4a49-966f-749fcbefbac4 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:53.999097 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1a431-e2b8-4a49-966f-749fcbefbac4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:53.999587 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1a431-e2b8-4a49-966f-749fcbefbac4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:54.000911 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1a431-e2b8-4a49-966f-749fcbefbac4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:54.012949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 640/1270] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:54.019348 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92da1d-41d1-4a42-aaf0-ac572825c87a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:54.052030 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:54.118342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92da1d-41d1-4a42-aaf0-ac572825c87a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:54.153212 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa5a9d-5670-474f-b63e-ecce91c55509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:54.154074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa5a9d-5670-474f-b63e-ecce91c55509 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:54.154362 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa5a9d-5670-474f-b63e-ecce91c55509 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:54.154639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa5a9d-5670-474f-b63e-ecce91c55509 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:54.161958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa5a9d-5670-474f-b63e-ecce91c55509 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:54.163721 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa5a9d-5670-474f-b63e-ecce91c55509 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:54.177084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-90fa5a9d-5670-474f-b63e-ecce91c55509 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:52:54.177084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:54.177084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:54.177084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:54.177084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:54.177084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:54.177084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:54.177084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:54.177084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:54.177084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:54.177084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:54.177084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:54.177084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:52:54.177084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:52:54.177084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:52:54.178263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:52:54.178263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:54.178263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:54.178263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:54.178263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:54.178263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:54.178263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:54.178263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:54.178263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:54.178263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:54.178263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:54.178263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:54.178263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:54.178263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:54.178263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:54.178263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:54.178263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:54.179372 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:52:54.179372 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:52:54.179372 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:52:54.179372 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:52:54.179372 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:52:54.179372 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:52:54.179372 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:52:54.179372 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:52:54.179372 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 631/1271] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:52:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:54.184097 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1a431-e2b8-4a49-966f-749fcbefbac4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:54.241107 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:54.281620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1a431-e2b8-4a49-966f-749fcbefbac4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:54.315181 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ad3a6-542b-40d7-a6f2-6897b31b3c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:54.315181 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ad3a6-542b-40d7-a6f2-6897b31b3c25 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:54.315181 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ad3a6-542b-40d7-a6f2-6897b31b3c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:54.315181 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ad3a6-542b-40d7-a6f2-6897b31b3c25 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:54.315624 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ad3a6-542b-40d7-a6f2-6897b31b3c25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:54.315940 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ad3a6-542b-40d7-a6f2-6897b31b3c25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:54.317377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ad3a6-542b-40d7-a6f2-6897b31b3c25 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:54.329976 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 641/1272] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:52:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:54.335866 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa5a9d-5670-474f-b63e-ecce91c55509 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:54.425180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:54.433401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa5a9d-5670-474f-b63e-ecce91c55509 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:54.467648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a79ab-e6c6-413e-8c11-5fbd1ab24f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:54.468553 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a79ab-e6c6-413e-8c11-5fbd1ab24f6a None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:54.468858 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a79ab-e6c6-413e-8c11-5fbd1ab24f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:54.469150 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a79ab-e6c6-413e-8c11-5fbd1ab24f6a None admin] PATH_INFO: `/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/c577fa8670e34449a80de0c8e0a15a8b/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:54.471059 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1a79ab-e6c6-413e-8c11-5fbd1ab24f6a None admin] RBAC: Authorizing `identity:create_grant(project_id=6003dba8ddd247e7842d28ea7ca5238c, user_id=c577fa8670e34449a80de0c8e0a15a8b, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:54.472523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1a79ab-e6c6-413e-8c11-5fbd1ab24f6a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:54.660188 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 632/1273] 10.222.0.69 () {58 vars in 1439 bytes} [Thu Jul 14 18:52:54 2022] PUT /identity/v3/projects/6003dba8ddd247e7842d28ea7ca5238c/users/c577fa8670e34449a80de0c8e0a15a8b/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:52:56.599035 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfefa5a-6a13-47e5-ad18-919cff4c6983 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:56.599637 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfefa5a-6a13-47e5-ad18-919cff4c6983 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:56.599637 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfefa5a-6a13-47e5-ad18-919cff4c6983 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:56.601053 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 642/1274] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:52:56.606143 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b363ca7-0bb4-4696-ad22-ba16745e843f None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:56.606538 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b363ca7-0bb4-4696-ad22-ba16745e843f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:56.606715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b363ca7-0bb4-4696-ad22-ba16745e843f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:56.623256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4b363ca7-0bb4-4696-ad22-ba16745e843f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:52:56.624444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4b363ca7-0bb4-4696-ad22-ba16745e843f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:52:56.690011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 633/1275] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:52:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:52:56.713100 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfefa5a-6a13-47e5-ad18-919cff4c6983 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:56.811761 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfefa5a-6a13-47e5-ad18-919cff4c6983 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:56.829648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:56.845838 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f9919-c49b-4de5-9dae-b1378322801f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:56.846655 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f9919-c49b-4de5-9dae-b1378322801f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:56.846941 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f9919-c49b-4de5-9dae-b1378322801f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:56.847244 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f9919-c49b-4de5-9dae-b1378322801f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:56.857403 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f9919-c49b-4de5-9dae-b1378322801f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 18:52:56.865371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-7b0f9919-c49b-4de5-9dae-b1378322801f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:52:56.865371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:56.865371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:56.865371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:56.865371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:56.865371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:56.865371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:56.865371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:56.865371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:56.865371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:56.865371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:56.865371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:56.865371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 18:52:56.865371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 18:52:56.865371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 18:52:56.865371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 18:52:56.866483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 18:52:56.866483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 18:52:56.866483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 18:52:56.866483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 18:52:56.866483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:56.866483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:56.866483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:56.866483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:56.866483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:56.866483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:56.866483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:56.866483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:56.866483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:56.866483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:56.866483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:56.866483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:56.867441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:56.867441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:56.867441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:56.867441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:56.867441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 18:52:56.867441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 18:52:56.867441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 18:52:56.867441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 18:52:56.867441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 18:52:56.867441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 18:52:56.867441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 18:52:56.867441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:52:56.867441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 643/1276] 10.222.0.69 () {56 vars in 1096 bytes} [Thu Jul 14 18:52:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:56.872712 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b363ca7-0bb4-4696-ad22-ba16745e843f None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:56.954720 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:56.971918 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b363ca7-0bb4-4696-ad22-ba16745e843f None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:57.006104 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63335159-e364-43b0-b4dd-80cbb07027a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:57.006785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63335159-e364-43b0-b4dd-80cbb07027a1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:57.007075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63335159-e364-43b0-b4dd-80cbb07027a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:57.007388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63335159-e364-43b0-b4dd-80cbb07027a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:57.008149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63335159-e364-43b0-b4dd-80cbb07027a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:57.008463 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63335159-e364-43b0-b4dd-80cbb07027a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:57.009845 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63335159-e364-43b0-b4dd-80cbb07027a1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:57.018483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 634/1277] 10.222.0.69 () {56 vars in 1099 bytes} [Thu Jul 14 18:52:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:57.023841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f9919-c49b-4de5-9dae-b1378322801f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:57.123836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f9919-c49b-4de5-9dae-b1378322801f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:57.157716 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158c7c1-5468-41c9-9c07-20571c7ae637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:57.158507 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158c7c1-5468-41c9-9c07-20571c7ae637 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:57.158793 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158c7c1-5468-41c9-9c07-20571c7ae637 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:57.159071 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158c7c1-5468-41c9-9c07-20571c7ae637 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:57.170110 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158c7c1-5468-41c9-9c07-20571c7ae637 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:57.171665 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158c7c1-5468-41c9-9c07-20571c7ae637 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:57.189355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-7158c7c1-5468-41c9-9c07-20571c7ae637 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:57.189355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:57.189355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:57.189355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:57.189355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:57.189355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:57.189355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:57.189355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:57.189355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:57.189355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:57.189355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:57.189355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:57.189355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:52:57.189355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:52:57.189355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:52:57.190533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:52:57.190533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:57.190533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:57.190533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:57.190533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:57.190533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:57.190533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:57.190533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:57.190533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:57.190533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:57.190533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:57.190533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:57.190533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:57.190533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:57.190533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:57.190533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:57.190533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:57.197143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:52:57.197143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:52:57.197143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:57.197143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:52:57.197143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 644/1278] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:57.197491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63335159-e364-43b0-b4dd-80cbb07027a1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:57.307290 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63335159-e364-43b0-b4dd-80cbb07027a1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:57.340066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30299153-5d5b-476b-a778-cfcfef9ded90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:57.340730 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30299153-5d5b-476b-a778-cfcfef9ded90 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:57.341016 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30299153-5d5b-476b-a778-cfcfef9ded90 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:57.341294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30299153-5d5b-476b-a778-cfcfef9ded90 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:57.342007 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30299153-5d5b-476b-a778-cfcfef9ded90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:57.342314 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30299153-5d5b-476b-a778-cfcfef9ded90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:57.344287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30299153-5d5b-476b-a778-cfcfef9ded90 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:57.356832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 635/1279] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:57.363421 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158c7c1-5468-41c9-9c07-20571c7ae637 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:57.393459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:57.459695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158c7c1-5468-41c9-9c07-20571c7ae637 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:57.492935 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3396c-1baf-46e2-8a73-a90389f97a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:57.493728 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3396c-1baf-46e2-8a73-a90389f97a57 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:57.494014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3396c-1baf-46e2-8a73-a90389f97a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:57.494290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3396c-1baf-46e2-8a73-a90389f97a57 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:57.501577 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3396c-1baf-46e2-8a73-a90389f97a57 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:57.503016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3396c-1baf-46e2-8a73-a90389f97a57 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:57.517032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c9d3396c-1baf-46e2-8a73-a90389f97a57 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 18:52:57.517032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:57.517032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:57.517032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:57.517032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:57.517032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:57.517032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:57.517032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:57.517032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:57.517032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:57.517032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:57.517032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:57.517032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 18:52:57.517032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 18:52:57.517032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 18:52:57.517032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 18:52:57.519907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:57.519907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:57.519907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 18:52:57.519907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:52:57.519907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 18:52:57.519907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 18:52:57.519907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:57.519907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:57.519907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:57.519907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:57.519907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:57.519907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:57.519907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:57.519907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:57.519907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:57.519907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:57.519907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:57.519907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:57.521090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:57.521090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:57.521090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 18:52:57.521090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 18:52:57.521090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 18:52:57.521090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 18:52:57.521090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 18:52:57.521090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 18:52:57.521090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 18:52:57.521090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:52:57.521090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 645/1280] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:57.524008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30299153-5d5b-476b-a778-cfcfef9ded90 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:57.528066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:57.625680 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30299153-5d5b-476b-a778-cfcfef9ded90 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:57.660390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55859b93-1a70-48c4-8f85-d5fb5951dfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:57.660992 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55859b93-1a70-48c4-8f85-d5fb5951dfe4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:57.661281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55859b93-1a70-48c4-8f85-d5fb5951dfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:57.661557 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55859b93-1a70-48c4-8f85-d5fb5951dfe4 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:57.662379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55859b93-1a70-48c4-8f85-d5fb5951dfe4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:57.662716 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55859b93-1a70-48c4-8f85-d5fb5951dfe4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:57.664246 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55859b93-1a70-48c4-8f85-d5fb5951dfe4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:57.681303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 636/1281] 10.222.0.69 () {56 vars in 1143 bytes} [Thu Jul 14 18:52:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:57.687308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3396c-1baf-46e2-8a73-a90389f97a57 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:57.775944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:57.786054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3396c-1baf-46e2-8a73-a90389f97a57 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:57.817675 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4b90a-492c-48ec-ad74-320cecd1511b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:57.818314 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4b90a-492c-48ec-ad74-320cecd1511b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:57.818598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4b90a-492c-48ec-ad74-320cecd1511b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:57.818870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4b90a-492c-48ec-ad74-320cecd1511b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:57.882000 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4b90a-492c-48ec-ad74-320cecd1511b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:57.883461 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4b90a-492c-48ec-ad74-320cecd1511b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:57.970344 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:57.980218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 646/1282] 10.222.0.69 () {58 vars in 1282 bytes} [Thu Jul 14 18:52:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 18:52:57.986035 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55859b93-1a70-48c4-8f85-d5fb5951dfe4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:58.049065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:58.087088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55859b93-1a70-48c4-8f85-d5fb5951dfe4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:58.118943 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1013ef-7ca8-4cd8-a174-fb84180801db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:58.119757 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1013ef-7ca8-4cd8-a174-fb84180801db None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:58.120060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1013ef-7ca8-4cd8-a174-fb84180801db None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:58.120346 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1013ef-7ca8-4cd8-a174-fb84180801db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:58.131526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1013ef-7ca8-4cd8-a174-fb84180801db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:58.132975 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1013ef-7ca8-4cd8-a174-fb84180801db None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:58.151608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-9a1013ef-7ca8-4cd8-a174-fb84180801db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:58.151608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:58.151608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:58.151608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:58.151608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:58.151608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:58.151608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:58.151608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:58.151608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:58.151608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:58.151608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:58.151608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:58.151608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 18:52:58.151608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 18:52:58.151608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 18:52:58.155900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 18:52:58.155900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:58.155900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:58.155900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:58.155900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:58.155900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:58.155900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:58.155900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:58.155900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:58.155900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:58.155900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:58.155900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:58.155900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:58.155900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:58.155900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:58.155900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:58.155900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:58.157099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 18:52:58.157099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 18:52:58.157099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 18:52:58.157099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:52:58.157099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 637/1283] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 18:52:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:58.159204 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4b90a-492c-48ec-ad74-320cecd1511b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:58.256043 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4b90a-492c-48ec-ad74-320cecd1511b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:58.289514 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d310f-a1d5-430d-8ad1-e5e083e0db6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:58.290198 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d310f-a1d5-430d-8ad1-e5e083e0db6b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:58.290481 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d310f-a1d5-430d-8ad1-e5e083e0db6b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:58.290760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d310f-a1d5-430d-8ad1-e5e083e0db6b None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:58.291503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d310f-a1d5-430d-8ad1-e5e083e0db6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:58.293193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d310f-a1d5-430d-8ad1-e5e083e0db6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:58.293193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d310f-a1d5-430d-8ad1-e5e083e0db6b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:58.304384 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 647/1284] 10.222.0.69 () {56 vars in 1109 bytes} [Thu Jul 14 18:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:58.310057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1013ef-7ca8-4cd8-a174-fb84180801db None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:58.410378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1013ef-7ca8-4cd8-a174-fb84180801db None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:58.424580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:58.444494 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de79e7-49b0-4511-b9fd-17a9e4b94446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:58.445289 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de79e7-49b0-4511-b9fd-17a9e4b94446 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:58.445557 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de79e7-49b0-4511-b9fd-17a9e4b94446 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:58.445833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de79e7-49b0-4511-b9fd-17a9e4b94446 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:58.452327 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de79e7-49b0-4511-b9fd-17a9e4b94446 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:58.453756 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de79e7-49b0-4511-b9fd-17a9e4b94446 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:58.467745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-d2de79e7-49b0-4511-b9fd-17a9e4b94446 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:52:58.467745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:52:58.467745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:52:58.467745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:52:58.467745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:52:58.467745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:52:58.467745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:52:58.467745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:52:58.467745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:52:58.467745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:52:58.467745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:52:58.467745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:52:58.467745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 18:52:58.467745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 18:52:58.467745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 18:52:58.468771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 18:52:58.468771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:52:58.468771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:52:58.468771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:52:58.468771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:52:58.468771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:52:58.468771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:52:58.468771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:52:58.468771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jul 14 18:52:58.468771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:52:58.468771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:52:58.468771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:52:58.468771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:52:58.468771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:52:58.468771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:52:58.468771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:52:58.468771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:52:58.469810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 18:52:58.469810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 18:52:58.469810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 18:52:58.469810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 18:52:58.469810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 18:52:58.469810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 18:52:58.469810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 18:52:58.469810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jul 14 18:52:58.469810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 638/1285] 10.222.0.69 () {56 vars in 1111 bytes} [Thu Jul 14 18:52:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:52:58.474860 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d310f-a1d5-430d-8ad1-e5e083e0db6b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:58.525721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:52:58.574743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d310f-a1d5-430d-8ad1-e5e083e0db6b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:58.609051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bdfa17-ef0a-443d-9be7-992219c7619c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:58.610217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdfa17-ef0a-443d-9be7-992219c7619c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:58.610504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdfa17-ef0a-443d-9be7-992219c7619c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:58.610783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdfa17-ef0a-443d-9be7-992219c7619c None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:58.611971 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bdfa17-ef0a-443d-9be7-992219c7619c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:58.612288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bdfa17-ef0a-443d-9be7-992219c7619c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:58.613737 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bdfa17-ef0a-443d-9be7-992219c7619c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:58.626189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 648/1286] 10.222.0.69 () {56 vars in 1148 bytes} [Thu Jul 14 18:52:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:52:58.632345 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de79e7-49b0-4511-b9fd-17a9e4b94446 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:52:58.729533 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de79e7-49b0-4511-b9fd-17a9e4b94446 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:52:58.762657 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca4aa6-0438-4f8e-b2fa-58f5accc60e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:52:58.763332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca4aa6-0438-4f8e-b2fa-58f5accc60e4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:52:58.763622 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca4aa6-0438-4f8e-b2fa-58f5accc60e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:52:58.763901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca4aa6-0438-4f8e-b2fa-58f5accc60e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:52:58.764734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca4aa6-0438-4f8e-b2fa-58f5accc60e4 None admin] RBAC: Adding query filter params (role.id=27fc14631c494ca0b2dc7f708be016c2, scope.project.id=6003dba8ddd247e7842d28ea7ca5238c, user.id=c577fa8670e34449a80de0c8e0a15a8b) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:52:58.765046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca4aa6-0438-4f8e-b2fa-58f5accc60e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:52:58.766690 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca4aa6-0438-4f8e-b2fa-58f5accc60e4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:52:58.772453 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 639/1287] 10.222.0.69 () {56 vars in 1374 bytes} [Thu Jul 14 18:52:58 2022] GET /identity/v3/role_assignments?user.id=c577fa8670e34449a80de0c8e0a15a8b&scope.project.id=6003dba8ddd247e7842d28ea7ca5238c&role.id=27fc14631c494ca0b2dc7f708be016c2 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:53:00.778218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba9f80-8699-4e6c-bb6a-dc7ca6718c71 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:00.778749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba9f80-8699-4e6c-bb6a-dc7ca6718c71 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:00.778749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba9f80-8699-4e6c-bb6a-dc7ca6718c71 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:00.780213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 649/1288] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:53:00.785935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40ecfe-b2c2-45ea-8e54-b665b749d9b9 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:00.786353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40ecfe-b2c2-45ea-8e54-b665b749d9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:00.786557 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40ecfe-b2c2-45ea-8e54-b665b749d9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:00.803754 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0f40ecfe-b2c2-45ea-8e54-b665b749d9b9 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:53:00.804933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0f40ecfe-b2c2-45ea-8e54-b665b749d9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:53:00.833834 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:53:00.869623 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 640/1289] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:53:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:53:00.892001 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba9f80-8699-4e6c-bb6a-dc7ca6718c71 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:53:00.993446 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba9f80-8699-4e6c-bb6a-dc7ca6718c71 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:53:01.004704 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:53:01.027347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cc08f-4b17-4581-a7fc-7b10826f8fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:53:01.028516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cc08f-4b17-4581-a7fc-7b10826f8fcb None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:01.028746 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cc08f-4b17-4581-a7fc-7b10826f8fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:01.029017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cc08f-4b17-4581-a7fc-7b10826f8fcb None admin] PATH_INFO: `/v3/services/placement` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:01.029653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221cc08f-4b17-4581-a7fc-7b10826f8fcb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:53:01.031017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221cc08f-4b17-4581-a7fc-7b10826f8fcb None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:53:01.042639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-221cc08f-4b17-4581-a7fc-7b10826f8fcb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 18:53:01.042639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:53:01.042639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:53:01.042639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:53:01.042639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:53:01.042639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:53:01.042639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:53:01.042639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:53:01.042639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 18:53:01.042639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:53:01.042639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:53:01.042639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:53:01.042639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:53:01.042639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:53:01.042639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:53:01.042639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:53:01.042639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:53:01.043920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:53:01.043920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:53:01.043920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:53:01.043920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:53:01.043920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:53:01.043920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:53:01.043920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:53:01.043920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:53:01.043920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:53:01.043920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:53:01.043920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:53:01.043920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:53:01.043920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:53:01.043920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:53:01.043920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:53:01.043920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:53:01.045023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:53:01.045023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:53:01.045023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:53:01.045023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:53:01.045023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:53:01.045023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:53:01.045023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:53:01.045023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:53:01.045023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:53:01.045023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:53:01.045023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:53:01.045023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:53:01.045023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 18:53:01.045023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:53:01.045023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 650/1290] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 14 18:53:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:53:01.049881 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40ecfe-b2c2-45ea-8e54-b665b749d9b9 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:53:01.147218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40ecfe-b2c2-45ea-8e54-b665b749d9b9 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:53:01.180555 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb334cc-17af-4628-835f-de435af1502b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:53:01.181209 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb334cc-17af-4628-835f-de435af1502b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:01.181494 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb334cc-17af-4628-835f-de435af1502b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:01.181771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb334cc-17af-4628-835f-de435af1502b None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:01.182464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb334cc-17af-4628-835f-de435af1502b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:53:01.182816 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb334cc-17af-4628-835f-de435af1502b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:53:01.184219 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb334cc-17af-4628-835f-de435af1502b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:53:01.188982 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 641/1291] 10.222.0.69 () {56 vars in 1116 bytes} [Thu Jul 14 18:53:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:53:01.195235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cc08f-4b17-4581-a7fc-7b10826f8fcb None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:53:01.292848 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cc08f-4b17-4581-a7fc-7b10826f8fcb None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:53:01.325842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8c070-45a5-413e-a356-b4bdce3d57ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:53:01.326526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8c070-45a5-413e-a356-b4bdce3d57ad None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:01.326809 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8c070-45a5-413e-a356-b4bdce3d57ad None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:01.327083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8c070-45a5-413e-a356-b4bdce3d57ad None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:01.328349 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8c070-45a5-413e-a356-b4bdce3d57ad None admin] RBAC: Adding query filter params (type=placement) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:53:01.328654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8c070-45a5-413e-a356-b4bdce3d57ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:53:01.330019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8c070-45a5-413e-a356-b4bdce3d57ad None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:53:01.334435 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 651/1292] 10.222.0.69 () {56 vars in 1116 bytes} [Thu Jul 14 18:53:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:53:03.306446 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1293f29-a89d-4f1b-b722-5494bb4a6f5d None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:03.306974 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1293f29-a89d-4f1b-b722-5494bb4a6f5d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:03.306974 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1293f29-a89d-4f1b-b722-5494bb4a6f5d None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:03.308473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 642/1293] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:53:03.314059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d20c5-af1e-4864-93ef-bf6a9180457a None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:03.314541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d20c5-af1e-4864-93ef-bf6a9180457a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:03.314630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d20c5-af1e-4864-93ef-bf6a9180457a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:03.331259 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2b5d20c5-af1e-4864-93ef-bf6a9180457a None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:53:03.332613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2b5d20c5-af1e-4864-93ef-bf6a9180457a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:53:03.362689 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:53:03.399732 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 652/1294] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:53:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:53:03.423227 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1293f29-a89d-4f1b-b722-5494bb4a6f5d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:53:03.453943 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:53:03.523366 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1293f29-a89d-4f1b-b722-5494bb4a6f5d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:53:03.557755 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba99c29c-dc94-4c68-be2b-076ae8f9342e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:53:03.558355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99c29c-dc94-4c68-be2b-076ae8f9342e None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:03.558644 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99c29c-dc94-4c68-be2b-076ae8f9342e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:03.558925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99c29c-dc94-4c68-be2b-076ae8f9342e None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:03.559699 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba99c29c-dc94-4c68-be2b-076ae8f9342e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:53:03.561065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba99c29c-dc94-4c68-be2b-076ae8f9342e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:53:03.663290 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 643/1295] 10.222.0.69 () {60 vars in 1141 bytes} [Thu Jul 14 18:53:03 2022] POST /identity/v3/services => generated 246 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:53:05.682406 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d13272-c3ed-4c11-891b-7f7abedd39f4 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:05.682961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d13272-c3ed-4c11-891b-7f7abedd39f4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:05.682961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d13272-c3ed-4c11-891b-7f7abedd39f4 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:05.684404 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 653/1296] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:53:05.689982 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a98bd-8f77-4039-b411-8cead71ec1f3 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:05.690386 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a98bd-8f77-4039-b411-8cead71ec1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:05.690552 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a98bd-8f77-4039-b411-8cead71ec1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:05.707763 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-fc2a98bd-8f77-4039-b411-8cead71ec1f3 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:53:05.708954 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-fc2a98bd-8f77-4039-b411-8cead71ec1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:53:05.780798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 644/1297] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:53:05 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:53:05.802618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d13272-c3ed-4c11-891b-7f7abedd39f4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:53:05.912978 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d13272-c3ed-4c11-891b-7f7abedd39f4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:53:05.953463 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99218f0-6038-4a4a-80fb-dae88a0ed7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:53:05.954220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99218f0-6038-4a4a-80fb-dae88a0ed7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:05.954504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99218f0-6038-4a4a-80fb-dae88a0ed7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:05.954782 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99218f0-6038-4a4a-80fb-dae88a0ed7a5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:05.955458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99218f0-6038-4a4a-80fb-dae88a0ed7a5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:53:05.956845 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99218f0-6038-4a4a-80fb-dae88a0ed7a5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:53:05.968858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-d99218f0-6038-4a4a-80fb-dae88a0ed7a5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 18:53:05.968858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:53:05.968858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:53:05.968858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:53:05.968858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:53:05.968858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:53:05.968858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:53:05.968858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:53:05.968858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 18:53:05.968858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:53:05.968858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:53:05.968858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:53:05.968858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:53:05.968858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:53:05.968858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:53:05.968858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:53:05.968858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:53:05.973377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:53:05.973377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:53:05.973377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:53:05.973377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:53:05.973377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:53:05.973377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:53:05.973377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:53:05.973377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:53:05.973377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:53:05.973377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:53:05.973377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:53:05.973377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:53:05.973377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:53:05.973377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:53:05.973377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:53:05.973377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:53:05.974432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:53:05.974432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:53:05.974432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:53:05.974432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:53:05.974432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:53:05.974432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:53:05.974432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:53:05.974432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:53:05.974432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:53:05.974432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:53:05.974432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:53:05.974432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:53:05.974432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 18:53:05.974432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:53:05.974432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 654/1298] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 14 18:53:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:53:05.976290 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a98bd-8f77-4039-b411-8cead71ec1f3 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:53:05.997869 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:53:06.077316 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a98bd-8f77-4039-b411-8cead71ec1f3 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:53:06.109794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f2135-7a4a-43c6-9008-b95a0c6d0eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:53:06.110454 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f2135-7a4a-43c6-9008-b95a0c6d0eda None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:06.110755 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f2135-7a4a-43c6-9008-b95a0c6d0eda None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:06.111015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f2135-7a4a-43c6-9008-b95a0c6d0eda None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:06.111748 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f2135-7a4a-43c6-9008-b95a0c6d0eda None admin] RBAC: Adding query filter params (name=placement) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:53:06.112054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f2135-7a4a-43c6-9008-b95a0c6d0eda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:53:06.113418 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f2135-7a4a-43c6-9008-b95a0c6d0eda None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:53:06.118414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 645/1299] 10.222.0.69 () {56 vars in 1116 bytes} [Thu Jul 14 18:53:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:53:06.125506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99218f0-6038-4a4a-80fb-dae88a0ed7a5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:53:06.156595 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:53:06.223732 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99218f0-6038-4a4a-80fb-dae88a0ed7a5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:53:06.257521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2354b-cb01-4288-bf7e-18c80ce234b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:53:06.258171 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2354b-cb01-4288-bf7e-18c80ce234b8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:06.258458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2354b-cb01-4288-bf7e-18c80ce234b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:06.258737 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2354b-cb01-4288-bf7e-18c80ce234b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:06.259558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2354b-cb01-4288-bf7e-18c80ce234b8 None admin] RBAC: Adding query filter params (interface=public, service_id=276dacf3a4a249ddba82b37b7cf89b8f, region_id=RegionOne) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:53:06.259872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2354b-cb01-4288-bf7e-18c80ce234b8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:53:06.261264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2354b-cb01-4288-bf7e-18c80ce234b8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:53:06.266219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 655/1300] 10.222.0.69 () {56 vars in 1251 bytes} [Thu Jul 14 18:53:06 2022] GET /identity/v3/endpoints?service_id=276dacf3a4a249ddba82b37b7cf89b8f&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:53:06.271954 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f2135-7a4a-43c6-9008-b95a0c6d0eda None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:53:06.352553 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:53:06.370341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f2135-7a4a-43c6-9008-b95a0c6d0eda None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:53:06.403285 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a482619-560f-40aa-a5b0-bc9566a13bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:53:06.403957 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a482619-560f-40aa-a5b0-bc9566a13bdc None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:06.404247 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a482619-560f-40aa-a5b0-bc9566a13bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:06.404526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a482619-560f-40aa-a5b0-bc9566a13bdc None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:06.405213 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a482619-560f-40aa-a5b0-bc9566a13bdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:53:06.407109 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a482619-560f-40aa-a5b0-bc9566a13bdc None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:53:06.414091 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 646/1301] 10.222.0.69 () {56 vars in 1087 bytes} [Thu Jul 14 18:53:06 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:53:08.420159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985bd0c5-18c2-4524-8a2b-09214fce7996 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:08.420832 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985bd0c5-18c2-4524-8a2b-09214fce7996 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:08.420832 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985bd0c5-18c2-4524-8a2b-09214fce7996 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:08.422129 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 656/1302] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:53:08.427345 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e4291-9f17-4b67-9710-92e430a992b3 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:08.427783 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e4291-9f17-4b67-9710-92e430a992b3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:08.427909 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e4291-9f17-4b67-9710-92e430a992b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:08.445170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-134e4291-9f17-4b67-9710-92e430a992b3 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:53:08.446347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-134e4291-9f17-4b67-9710-92e430a992b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:53:08.512571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 647/1303] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:53:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 18:53:08.536158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985bd0c5-18c2-4524-8a2b-09214fce7996 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:53:08.629552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:53:08.636101 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985bd0c5-18c2-4524-8a2b-09214fce7996 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:53:08.669323 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf240ab-5bb9-442c-aca9-046c04028a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:53:08.670109 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf240ab-5bb9-442c-aca9-046c04028a56 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:08.670439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf240ab-5bb9-442c-aca9-046c04028a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:08.670658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf240ab-5bb9-442c-aca9-046c04028a56 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:08.671348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf240ab-5bb9-442c-aca9-046c04028a56 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:53:08.672710 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf240ab-5bb9-442c-aca9-046c04028a56 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:53:08.684140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-bbf240ab-5bb9-442c-aca9-046c04028a56 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 18:53:08.684140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:53:08.684140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 18:53:08.684140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 18:53:08.684140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 18:53:08.684140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 18:53:08.684140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 18:53:08.684140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:53:08.684140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 18:53:08.684140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:53:08.684140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 18:53:08.684140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:53:08.684140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 18:53:08.684140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 18:53:08.684140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 18:53:08.684140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 18:53:08.684140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 18:53:08.685333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 18:53:08.685333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 18:53:08.685333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 18:53:08.685333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 18:53:08.685333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 18:53:08.685333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 18:53:08.685333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 18:53:08.685333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 18:53:08.685333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 18:53:08.685333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 18:53:08.685333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 18:53:08.685333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 18:53:08.685333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 18:53:08.685333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 18:53:08.685333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 18:53:08.685333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 18:53:08.687222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 18:53:08.687222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jul 14 18:53:08.687222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 18:53:08.687222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jul 14 18:53:08.687222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 18:53:08.687222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 18:53:08.687222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 18:53:08.687222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jul 14 18:53:08.687222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 18:53:08.687222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jul 14 18:53:08.687222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 18:53:08.687222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 18:53:08.687222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 18:53:08.687222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jul 14 18:53:08.687222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 657/1304] 10.222.0.69 () {56 vars in 1117 bytes} [Thu Jul 14 18:53:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 18:53:08.692001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e4291-9f17-4b67-9710-92e430a992b3 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:53:08.766577 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:53:08.788422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e4291-9f17-4b67-9710-92e430a992b3 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:53:08.820998 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c76f2-92ea-4e03-8a8e-53e7b0d3297f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:53:08.821654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c76f2-92ea-4e03-8a8e-53e7b0d3297f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:08.821942 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c76f2-92ea-4e03-8a8e-53e7b0d3297f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:08.822220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c76f2-92ea-4e03-8a8e-53e7b0d3297f None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:08.822920 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c76f2-92ea-4e03-8a8e-53e7b0d3297f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:53:08.823256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c76f2-92ea-4e03-8a8e-53e7b0d3297f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:53:08.824624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c76f2-92ea-4e03-8a8e-53e7b0d3297f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:53:08.830297 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 648/1305] 10.222.0.69 () {56 vars in 1116 bytes} [Thu Jul 14 18:53:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:53:08.836203 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf240ab-5bb9-442c-aca9-046c04028a56 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:53:08.933080 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf240ab-5bb9-442c-aca9-046c04028a56 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:53:08.966344 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eae62e-fc23-4bb4-bb7d-6ebf85e1d356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:53:08.966995 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eae62e-fc23-4bb4-bb7d-6ebf85e1d356 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:53:08.967314 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eae62e-fc23-4bb4-bb7d-6ebf85e1d356 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:53:08.967589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eae62e-fc23-4bb4-bb7d-6ebf85e1d356 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:53:08.968357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eae62e-fc23-4bb4-bb7d-6ebf85e1d356 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:53:08.969778 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eae62e-fc23-4bb4-bb7d-6ebf85e1d356 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:53:09.075722 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 658/1306] 10.222.0.69 () {60 vars in 1144 bytes} [Thu Jul 14 18:53:08 2022] POST /identity/v3/endpoints => generated 331 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:53:18.858013 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=30 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:53:19.071061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:14.577303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb7d4d-266c-43e4-be34-15faa4384c1c None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:14.578259 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb7d4d-266c-43e4-be34-15faa4384c1c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:14.578690 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb7d4d-266c-43e4-be34-15faa4384c1c None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:14.586295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e8b9f-6e76-4f5b-a2c0-a04c54a0a4ce None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:14.587248 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e8b9f-6e76-4f5b-a2c0-a04c54a0a4ce None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:14.587681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e8b9f-6e76-4f5b-a2c0-a04c54a0a4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:14.591632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 649/1307] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:59:14 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:59:14.691327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ec1e8b9f-6e76-4f5b-a2c0-a04c54a0a4ce None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:59:14.703911 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ec1e8b9f-6e76-4f5b-a2c0-a04c54a0a4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:14.875272 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 659/1308] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:59:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 18:59:14.927598 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8416e9-8596-4517-a8eb-db5b019a12e8 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:14.928315 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8416e9-8596-4517-a8eb-db5b019a12e8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:14.928863 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8416e9-8596-4517-a8eb-db5b019a12e8 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:14.931542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 650/1309] 10.222.0.69 () {54 vars in 964 bytes} [Thu Jul 14 18:59:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:59:14.937560 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444635ec-8efd-451a-90a3-be4989e195c9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:14.938214 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444635ec-8efd-451a-90a3-be4989e195c9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:14.938718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444635ec-8efd-451a-90a3-be4989e195c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:15.116755 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-444635ec-8efd-451a-90a3-be4989e195c9 None None] MFA Rules not processed for user `74b8c75b9cce442b8c430fab6a3fea17`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:59:15.118654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-444635ec-8efd-451a-90a3-be4989e195c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:15.158847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:15.300647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 660/1310] 10.222.0.69 () {58 vars in 1061 bytes} [Thu Jul 14 18:59:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:59:15.305348 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8416e9-8596-4517-a8eb-db5b019a12e8 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:59:15.307101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4f8416e9-8596-4517-a8eb-db5b019a12e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:15.531212 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8416e9-8596-4517-a8eb-db5b019a12e8 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:59:15.634321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa82d4a-0631-4622-bb59-19ed2d5a606b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:59:15.634987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa82d4a-0631-4622-bb59-19ed2d5a606b service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:15.635279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa82d4a-0631-4622-bb59-19ed2d5a606b service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:15.635546 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa82d4a-0631-4622-bb59-19ed2d5a606b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:15.637199 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-baa82d4a-0631-4622-bb59-19ed2d5a606b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:15.698383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:15.768538 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa82d4a-0631-4622-bb59-19ed2d5a606b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:59:15.770524 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa82d4a-0631-4622-bb59-19ed2d5a606b service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:59:15.798981 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 651/1311] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 18:59:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:59:18.195271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df3b4b-e982-417f-9efd-8d0bfe2b34da None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:18.195957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df3b4b-e982-417f-9efd-8d0bfe2b34da None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:18.195957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df3b4b-e982-417f-9efd-8d0bfe2b34da None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:18.197243 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 661/1312] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:59:18.202644 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f4d95-e991-40e7-a674-a084b8037714 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:18.203112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f4d95-e991-40e7-a674-a084b8037714 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:18.203382 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f4d95-e991-40e7-a674-a084b8037714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:18.287159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-638f4d95-e991-40e7-a674-a084b8037714 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:59:18.288353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-638f4d95-e991-40e7-a674-a084b8037714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:18.391199 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 652/1313] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:59:18.431510 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df3b4b-e982-417f-9efd-8d0bfe2b34da None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:59:18.433361 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-93df3b4b-e982-417f-9efd-8d0bfe2b34da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:18.559928 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df3b4b-e982-417f-9efd-8d0bfe2b34da None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:59:18.637032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:18.640196 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be717636-da44-449e-bd34-1cc9d2872b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:59:18.640856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be717636-da44-449e-bd34-1cc9d2872b22 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:18.641142 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be717636-da44-449e-bd34-1cc9d2872b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:18.641416 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be717636-da44-449e-bd34-1cc9d2872b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:18.643084 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-be717636-da44-449e-bd34-1cc9d2872b22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:18.756866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be717636-da44-449e-bd34-1cc9d2872b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:59:18.758325 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be717636-da44-449e-bd34-1cc9d2872b22 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:59:18.780329 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 662/1314] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 18:59:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 18:59:21.329121 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473251a-75f9-4355-af23-0f7cf5ac727f None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:21.329665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473251a-75f9-4355-af23-0f7cf5ac727f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:21.329665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473251a-75f9-4355-af23-0f7cf5ac727f None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:21.331061 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 653/1315] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:59:21.336538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b901e34-68ad-4b36-9ae0-21c1add03fd4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:21.337025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b901e34-68ad-4b36-9ae0-21c1add03fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:21.337123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b901e34-68ad-4b36-9ae0-21c1add03fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:21.408270 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9b901e34-68ad-4b36-9ae0-21c1add03fd4 None None] MFA Rules not processed for user `e383b5a76d134501a90ca526cde31f54`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:59:21.409526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9b901e34-68ad-4b36-9ae0-21c1add03fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:21.466630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:21.513407 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 663/1316] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:59:21 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:59:21.569439 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a9e6f8-d277-44d7-bc58-d7cc976cc351 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:21.569734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a9e6f8-d277-44d7-bc58-d7cc976cc351 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:21.570020 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a9e6f8-d277-44d7-bc58-d7cc976cc351 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:21.571435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 654/1317] 10.222.0.69 () {54 vars in 964 bytes} [Thu Jul 14 18:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:59:21.577673 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e08664-8989-4758-9524-1b8f418706e1 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:21.577959 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e08664-8989-4758-9524-1b8f418706e1 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:21.578237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e08664-8989-4758-9524-1b8f418706e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:21.656372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-04e08664-8989-4758-9524-1b8f418706e1 None None] MFA Rules not processed for user `74b8c75b9cce442b8c430fab6a3fea17`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:59:21.657503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-04e08664-8989-4758-9524-1b8f418706e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:21.736546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 664/1318] 10.222.0.69 () {58 vars in 1061 bytes} [Thu Jul 14 18:59:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:59:21.746235 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a9e6f8-d277-44d7-bc58-d7cc976cc351 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:59:21.747820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-62a9e6f8-d277-44d7-bc58-d7cc976cc351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:21.757959 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:21.858652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a9e6f8-d277-44d7-bc58-d7cc976cc351 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:59:21.942132 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd556ec5-d7b8-4c70-9921-04ee76ec491c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:59:21.942694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd556ec5-d7b8-4c70-9921-04ee76ec491c service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:21.942983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd556ec5-d7b8-4c70-9921-04ee76ec491c service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:21.943292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd556ec5-d7b8-4c70-9921-04ee76ec491c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:21.944966 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-bd556ec5-d7b8-4c70-9921-04ee76ec491c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:22.010745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:22.059949 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd556ec5-d7b8-4c70-9921-04ee76ec491c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:59:22.061957 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd556ec5-d7b8-4c70-9921-04ee76ec491c service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:59:22.081731 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 655/1319] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 18:59:21 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:59:25.496858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5d1d5-cb71-44a6-ae4f-87f07189f0bc None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:25.497406 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5d1d5-cb71-44a6-ae4f-87f07189f0bc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:25.497406 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5d1d5-cb71-44a6-ae4f-87f07189f0bc None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:25.499542 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 665/1320] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:59:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:59:25.505510 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e5c9a-5aee-4660-ac43-9d8ae27b9e32 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:25.505973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e5c9a-5aee-4660-ac43-9d8ae27b9e32 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:25.506301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e5c9a-5aee-4660-ac43-9d8ae27b9e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:25.573464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-100e5c9a-5aee-4660-ac43-9d8ae27b9e32 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:59:25.574698 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-100e5c9a-5aee-4660-ac43-9d8ae27b9e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:25.676973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 656/1321] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:59:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:59:25.719817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5d1d5-cb71-44a6-ae4f-87f07189f0bc None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:59:25.721414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-57e5d1d5-cb71-44a6-ae4f-87f07189f0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:25.784438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:25.843273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5d1d5-cb71-44a6-ae4f-87f07189f0bc None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:59:25.921839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e90c3b-1936-4355-a35b-6438c69ce455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:59:25.922537 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e90c3b-1936-4355-a35b-6438c69ce455 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:25.922805 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e90c3b-1936-4355-a35b-6438c69ce455 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:25.923054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e90c3b-1936-4355-a35b-6438c69ce455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:25.924746 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-31e90c3b-1936-4355-a35b-6438c69ce455 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:26.053408 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:26.075063 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e90c3b-1936-4355-a35b-6438c69ce455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:59:26.076856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e90c3b-1936-4355-a35b-6438c69ce455 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:59:26.098819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 666/1322] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 18:59:25 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:59:29.504087 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f072b-b8f7-42c6-91ee-6f597ce38833 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:29.504619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f072b-b8f7-42c6-91ee-6f597ce38833 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:29.504619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f072b-b8f7-42c6-91ee-6f597ce38833 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:29.505953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 657/1323] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:59:29.511494 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79be211-6dab-4014-a0c8-6d8b49f08888 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:29.511992 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79be211-6dab-4014-a0c8-6d8b49f08888 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:29.512121 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79be211-6dab-4014-a0c8-6d8b49f08888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:29.581367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-a79be211-6dab-4014-a0c8-6d8b49f08888 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:59:29.582589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a79be211-6dab-4014-a0c8-6d8b49f08888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:29.703032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 667/1324] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:59:29.740238 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f072b-b8f7-42c6-91ee-6f597ce38833 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:59:29.741805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4b2f072b-b8f7-42c6-91ee-6f597ce38833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:29.763014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:29.856216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f072b-b8f7-42c6-91ee-6f597ce38833 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:59:29.934007 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6066ed-b30f-4ca7-a102-1800f40cd19f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:59:29.934670 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6066ed-b30f-4ca7-a102-1800f40cd19f service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:29.934958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6066ed-b30f-4ca7-a102-1800f40cd19f service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:29.935268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6066ed-b30f-4ca7-a102-1800f40cd19f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:29.937606 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-de6066ed-b30f-4ca7-a102-1800f40cd19f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:30.022642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:30.065343 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6066ed-b30f-4ca7-a102-1800f40cd19f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:59:30.066813 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6066ed-b30f-4ca7-a102-1800f40cd19f service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:59:30.090854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 658/1325] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 18:59:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:59:32.245002 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d11336-1310-4fe1-9b33-17910c31fb64 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:32.245559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d11336-1310-4fe1-9b33-17910c31fb64 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:32.245559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d11336-1310-4fe1-9b33-17910c31fb64 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:32.246851 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 668/1326] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:59:32.253555 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accfd04-55dd-4b08-a2a8-86b35cdef422 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:32.254088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accfd04-55dd-4b08-a2a8-86b35cdef422 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:32.254088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accfd04-55dd-4b08-a2a8-86b35cdef422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:32.323327 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0accfd04-55dd-4b08-a2a8-86b35cdef422 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:59:32.324485 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0accfd04-55dd-4b08-a2a8-86b35cdef422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:32.433754 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 659/1327] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:59:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:59:32.470461 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d11336-1310-4fe1-9b33-17910c31fb64 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:59:32.472180 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-63d11336-1310-4fe1-9b33-17910c31fb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:32.543850 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:32.590936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d11336-1310-4fe1-9b33-17910c31fb64 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:59:32.674581 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7819971-e913-4a96-877f-613c3d2a1c80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:59:32.677726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7819971-e913-4a96-877f-613c3d2a1c80 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:32.677726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7819971-e913-4a96-877f-613c3d2a1c80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:32.677726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7819971-e913-4a96-877f-613c3d2a1c80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:32.677726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e7819971-e913-4a96-877f-613c3d2a1c80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:32.794015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:32.796707 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7819971-e913-4a96-877f-613c3d2a1c80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:59:32.798199 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7819971-e913-4a96-877f-613c3d2a1c80 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:59:32.827368 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 669/1328] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 18:59:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:59:34.918524 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c74ffd-a4b5-49fc-9f96-aac6002b4f95 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:34.919105 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c74ffd-a4b5-49fc-9f96-aac6002b4f95 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:34.919105 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c74ffd-a4b5-49fc-9f96-aac6002b4f95 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:34.921054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 660/1329] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:59:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:59:34.927287 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23f825-a0e4-4b38-b5b9-8d867c940329 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:34.927780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23f825-a0e4-4b38-b5b9-8d867c940329 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:34.927993 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23f825-a0e4-4b38-b5b9-8d867c940329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:34.998744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ef23f825-a0e4-4b38-b5b9-8d867c940329 None None] MFA Rules not processed for user `e383b5a76d134501a90ca526cde31f54`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:59:34.999980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ef23f825-a0e4-4b38-b5b9-8d867c940329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:35.099510 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 670/1330] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:59:34 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:59:35.137552 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c74ffd-a4b5-49fc-9f96-aac6002b4f95 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:59:35.139748 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a8c74ffd-a4b5-49fc-9f96-aac6002b4f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:35.171128 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:35.245377 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c74ffd-a4b5-49fc-9f96-aac6002b4f95 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:59:35.321352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdaaa5-a3d9-4554-893f-aa9189bb02e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:59:35.322001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdaaa5-a3d9-4554-893f-aa9189bb02e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:35.322287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdaaa5-a3d9-4554-893f-aa9189bb02e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:35.322565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdaaa5-a3d9-4554-893f-aa9189bb02e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:35.324297 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-31cdaaa5-a3d9-4554-893f-aa9189bb02e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:35.401278 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:35.446948 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cdaaa5-a3d9-4554-893f-aa9189bb02e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:59:35.448473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cdaaa5-a3d9-4554-893f-aa9189bb02e5 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:59:35.472960 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 661/1331] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 18:59:35 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:59:37.996057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d22e9a-ed85-4a98-acdc-52533946ecdd None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:37.996877 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d22e9a-ed85-4a98-acdc-52533946ecdd None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:37.996877 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d22e9a-ed85-4a98-acdc-52533946ecdd None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:37.998032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 671/1332] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:59:38.003453 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba6c42-effe-44af-bcce-bd793ad5e2c7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:38.003929 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba6c42-effe-44af-bcce-bd793ad5e2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:38.004035 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba6c42-effe-44af-bcce-bd793ad5e2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:38.068415 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b1ba6c42-effe-44af-bcce-bd793ad5e2c7 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:59:38.069627 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b1ba6c42-effe-44af-bcce-bd793ad5e2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:38.176056 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 662/1333] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 18:59:38.213282 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d22e9a-ed85-4a98-acdc-52533946ecdd None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:59:38.215567 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c6d22e9a-ed85-4a98-acdc-52533946ecdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:38.285666 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:38.319113 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d22e9a-ed85-4a98-acdc-52533946ecdd None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:59:38.391799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bbc41-624b-4bed-9e2a-55e1d1198abe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:59:38.392447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bbc41-624b-4bed-9e2a-55e1d1198abe service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:38.392732 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bbc41-624b-4bed-9e2a-55e1d1198abe service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:38.393009 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bbc41-624b-4bed-9e2a-55e1d1198abe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:38.394633 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-179bbc41-624b-4bed-9e2a-55e1d1198abe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:38.502908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bbc41-624b-4bed-9e2a-55e1d1198abe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:59:38.504260 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bbc41-624b-4bed-9e2a-55e1d1198abe service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:59:38.515190 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:38.523520 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 672/1334] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 18:59:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:59:40.558654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dcef2-cfa9-4ca7-b3e9-c3058f51b7c3 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:40.559218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dcef2-cfa9-4ca7-b3e9-c3058f51b7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:40.559525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dcef2-cfa9-4ca7-b3e9-c3058f51b7c3 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:40.561878 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 663/1335] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:59:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:59:40.567115 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5c9db-27cd-4e03-a3bb-b8c2c67556cd None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:40.567566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5c9db-27cd-4e03-a3bb-b8c2c67556cd None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:40.567734 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5c9db-27cd-4e03-a3bb-b8c2c67556cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:40.636264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e4e5c9db-27cd-4e03-a3bb-b8c2c67556cd None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:59:40.637525 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e4e5c9db-27cd-4e03-a3bb-b8c2c67556cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:40.735916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 673/1336] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:59:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:59:40.776079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820dcef2-cfa9-4ca7-b3e9-c3058f51b7c3 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:59:40.777615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-820dcef2-cfa9-4ca7-b3e9-c3058f51b7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:40.817836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:40.882695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820dcef2-cfa9-4ca7-b3e9-c3058f51b7c3 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:59:40.956612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ff62b-af61-4bcb-84d5-0ae560effb5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:59:40.957263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ff62b-af61-4bcb-84d5-0ae560effb5c service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:40.957549 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ff62b-af61-4bcb-84d5-0ae560effb5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:40.957824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ff62b-af61-4bcb-84d5-0ae560effb5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:40.959481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-aa0ff62b-af61-4bcb-84d5-0ae560effb5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:41.043638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:41.070335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ff62b-af61-4bcb-84d5-0ae560effb5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:59:41.071827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ff62b-af61-4bcb-84d5-0ae560effb5c service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:59:41.090811 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 664/1337] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 18:59:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:59:43.663131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f58a07-dd8a-4594-bd03-8dd26fa1fcdc None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:43.663688 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f58a07-dd8a-4594-bd03-8dd26fa1fcdc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:43.663800 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f58a07-dd8a-4594-bd03-8dd26fa1fcdc None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:43.665115 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 674/1338] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:59:43.670950 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4374e-6a4f-4093-8f6f-7d7108c4ba76 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:43.671411 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4374e-6a4f-4093-8f6f-7d7108c4ba76 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:43.671524 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4374e-6a4f-4093-8f6f-7d7108c4ba76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:43.736052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d4b4374e-6a4f-4093-8f6f-7d7108c4ba76 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:59:43.737247 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d4b4374e-6a4f-4093-8f6f-7d7108c4ba76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:43.832005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 665/1339] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:59:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:59:43.868665 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f58a07-dd8a-4594-bd03-8dd26fa1fcdc None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:59:43.870244 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-96f58a07-dd8a-4594-bd03-8dd26fa1fcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:43.945146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:43.972448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f58a07-dd8a-4594-bd03-8dd26fa1fcdc None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:59:44.043836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f9738-319b-49d0-ae11-28a2001ca04a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:59:44.045111 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f9738-319b-49d0-ae11-28a2001ca04a service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:44.045392 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f9738-319b-49d0-ae11-28a2001ca04a service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:44.045664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f9738-319b-49d0-ae11-28a2001ca04a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:44.047322 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-087f9738-319b-49d0-ae11-28a2001ca04a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:44.155209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f9738-319b-49d0-ae11-28a2001ca04a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:59:44.156661 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f9738-319b-49d0-ae11-28a2001ca04a service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:59:44.174305 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 675/1340] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 18:59:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:59:48.356821 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705c750-ecec-4db4-a86a-1a30831fbde2 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:48.357279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705c750-ecec-4db4-a86a-1a30831fbde2 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:48.357446 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705c750-ecec-4db4-a86a-1a30831fbde2 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:48.359364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 666/1341] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:59:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:59:48.364893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73da1e-3939-4bb1-ab77-c8ed45f93640 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:48.365335 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73da1e-3939-4bb1-ab77-c8ed45f93640 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:48.365646 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73da1e-3939-4bb1-ab77-c8ed45f93640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:48.369836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:48.435564 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-df73da1e-3939-4bb1-ab77-c8ed45f93640 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:59:48.437362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-df73da1e-3939-4bb1-ab77-c8ed45f93640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:48.537822 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 676/1342] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:59:48.577038 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705c750-ecec-4db4-a86a-1a30831fbde2 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:59:48.578602 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5705c750-ecec-4db4-a86a-1a30831fbde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:48.627643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:48.684768 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705c750-ecec-4db4-a86a-1a30831fbde2 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:59:48.757675 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd8e7c-2078-4d4e-a7bc-d1ac8691a60c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:59:48.758323 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd8e7c-2078-4d4e-a7bc-d1ac8691a60c service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:48.758607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd8e7c-2078-4d4e-a7bc-d1ac8691a60c service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:48.758880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd8e7c-2078-4d4e-a7bc-d1ac8691a60c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:48.760552 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-58dd8e7c-2078-4d4e-a7bc-d1ac8691a60c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:48.852638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:48.869428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd8e7c-2078-4d4e-a7bc-d1ac8691a60c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:59:48.870882 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd8e7c-2078-4d4e-a7bc-d1ac8691a60c service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:59:48.889754 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 667/1343] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 18:59:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:59:51.715763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139308e-1e94-45a7-9927-708ba1dc2e5b None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:51.716352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139308e-1e94-45a7-9927-708ba1dc2e5b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:51.716352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139308e-1e94-45a7-9927-708ba1dc2e5b None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:51.718329 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 677/1344] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:59:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:59:51.724234 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea95a7-d1a8-4d8f-ae14-ea59c27003b6 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:51.724757 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea95a7-d1a8-4d8f-ae14-ea59c27003b6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:51.724961 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea95a7-d1a8-4d8f-ae14-ea59c27003b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:51.794629 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-77ea95a7-d1a8-4d8f-ae14-ea59c27003b6 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:59:51.796491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-77ea95a7-d1a8-4d8f-ae14-ea59c27003b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:51.894852 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 668/1345] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:59:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:59:51.933669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139308e-1e94-45a7-9927-708ba1dc2e5b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:59:51.935230 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7139308e-1e94-45a7-9927-708ba1dc2e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:52.027728 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:52.044774 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139308e-1e94-45a7-9927-708ba1dc2e5b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:59:52.118993 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e076e22-1e6a-4d9f-a5b1-b61398962a2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:59:52.119675 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e076e22-1e6a-4d9f-a5b1-b61398962a2d service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:52.119964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e076e22-1e6a-4d9f-a5b1-b61398962a2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:52.120241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e076e22-1e6a-4d9f-a5b1-b61398962a2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:52.121919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0e076e22-1e6a-4d9f-a5b1-b61398962a2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:52.233863 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e076e22-1e6a-4d9f-a5b1-b61398962a2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:59:52.235357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e076e22-1e6a-4d9f-a5b1-b61398962a2d service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:59:52.253319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 678/1346] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 18:59:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 18:59:56.461806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae378d-f49a-48c4-a21d-5cbda8605e4b None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:56.462336 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae378d-f49a-48c4-a21d-5cbda8605e4b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:56.462336 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae378d-f49a-48c4-a21d-5cbda8605e4b None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:56.463768 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 669/1347] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 18:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:59:56.469299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b18b9-2426-4637-a5ef-7277d46fd751 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:56.469775 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b18b9-2426-4637-a5ef-7277d46fd751 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:56.469923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b18b9-2426-4637-a5ef-7277d46fd751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:56.485306 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:56.541324 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5d5b18b9-2426-4637-a5ef-7277d46fd751 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:59:56.542496 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5d5b18b9-2426-4637-a5ef-7277d46fd751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:56.666640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 679/1348] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 18:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:59:56.707437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae378d-f49a-48c4-a21d-5cbda8605e4b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:59:56.709063 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c0ae378d-f49a-48c4-a21d-5cbda8605e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:56.773240 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:56.824619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae378d-f49a-48c4-a21d-5cbda8605e4b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:59:56.909395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a06e97-bc00-41cd-8788-817c691217e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:59:56.910072 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a06e97-bc00-41cd-8788-817c691217e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:59:56.910616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a06e97-bc00-41cd-8788-817c691217e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:59:56.910971 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a06e97-bc00-41cd-8788-817c691217e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:59:56.912742 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-30a06e97-bc00-41cd-8788-817c691217e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:59:57.036722 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:59:57.040821 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a06e97-bc00-41cd-8788-817c691217e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:59:57.042267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a06e97-bc00-41cd-8788-817c691217e6 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:59:57.063154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 670/1349] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 18:59:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:00:01.961549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52679f-bff9-492b-86b9-a7221da54341 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:01.962207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52679f-bff9-492b-86b9-a7221da54341 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:01.962910 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52679f-bff9-492b-86b9-a7221da54341 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:01.965165 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 680/1350] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 14 19:00:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:01.971475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffdf54-8c63-491d-be2f-69062d1b3c3d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:01.971941 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffdf54-8c63-491d-be2f-69062d1b3c3d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:01.972261 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffdf54-8c63-491d-be2f-69062d1b3c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:02.129489 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0dffdf54-8c63-491d-be2f-69062d1b3c3d None None] MFA Rules not processed for user `c577fa8670e34449a80de0c8e0a15a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:02.131794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0dffdf54-8c63-491d-be2f-69062d1b3c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:02.290829 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 671/1351] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 14 19:00:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:06.664759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=59 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:06.822494 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22526a-be1e-41ed-a759-e0370f6c0a11 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:06.822894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22526a-be1e-41ed-a759-e0370f6c0a11 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:06.823220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22526a-be1e-41ed-a759-e0370f6c0a11 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:06.824780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 681/1352] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 14 19:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:06.831030 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d5efd-f42c-491c-acd0-1c69baae818f None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:06.831967 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d5efd-f42c-491c-acd0-1c69baae818f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:06.832477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d5efd-f42c-491c-acd0-1c69baae818f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:06.897389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:06.920613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-e38d5efd-f42c-491c-acd0-1c69baae818f None None] MFA Rules not processed for user `c577fa8670e34449a80de0c8e0a15a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:06.921788 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e38d5efd-f42c-491c-acd0-1c69baae818f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:07.051463 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 672/1353] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 14 19:00:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:08.211985 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee0db3-1834-4dc8-afb9-fb708096f66b None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:08.212532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee0db3-1834-4dc8-afb9-fb708096f66b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:08.212532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee0db3-1834-4dc8-afb9-fb708096f66b None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:08.213920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 682/1354] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 14 19:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:08.219050 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d8c8c-1823-4785-a5c4-36a5cfa63942 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:08.219474 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d8c8c-1823-4785-a5c4-36a5cfa63942 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:08.219703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d8c8c-1823-4785-a5c4-36a5cfa63942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:08.294234 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-e29d8c8c-1823-4785-a5c4-36a5cfa63942 None None] MFA Rules not processed for user `c577fa8670e34449a80de0c8e0a15a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:08.295380 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e29d8c8c-1823-4785-a5c4-36a5cfa63942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:08.338277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:08.405495 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 673/1355] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 14 19:00:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:11.823780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bce528-8fb7-4f31-9ef1-a0a7082167c2 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:11.824305 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bce528-8fb7-4f31-9ef1-a0a7082167c2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:11.824389 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bce528-8fb7-4f31-9ef1-a0a7082167c2 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:11.825871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 683/1356] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:11.831825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb3218-322b-48f8-a97a-32ad4f32d859 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:11.832648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb3218-322b-48f8-a97a-32ad4f32d859 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:11.833599 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb3218-322b-48f8-a97a-32ad4f32d859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:11.916484 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3efb3218-322b-48f8-a97a-32ad4f32d859 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:11.917644 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3efb3218-322b-48f8-a97a-32ad4f32d859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:12.034314 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 674/1357] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:00:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:12.093985 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac495d8f-b549-49fc-aa3c-86c1bafb55ee None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:12.097818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac495d8f-b549-49fc-aa3c-86c1bafb55ee None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:12.098329 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac495d8f-b549-49fc-aa3c-86c1bafb55ee None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:12.100148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 684/1358] 10.222.0.69 () {54 vars in 961 bytes} [Thu Jul 14 19:00:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:12.112447 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b4c99-489a-4bcf-9e86-4a06af991125 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:12.113070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b4c99-489a-4bcf-9e86-4a06af991125 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:12.113519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b4c99-489a-4bcf-9e86-4a06af991125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:12.136941 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:12.246118 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7f8b4c99-489a-4bcf-9e86-4a06af991125 None None] MFA Rules not processed for user `8d27d6df5cb64697adebd961dfbec324`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:12.248823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7f8b4c99-489a-4bcf-9e86-4a06af991125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:12.402876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 675/1359] 10.222.0.69 () {58 vars in 1058 bytes} [Thu Jul 14 19:00:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:12.406001 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac495d8f-b549-49fc-aa3c-86c1bafb55ee None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:00:12.407758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ac495d8f-b549-49fc-aa3c-86c1bafb55ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:12.611319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac495d8f-b549-49fc-aa3c-86c1bafb55ee None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:00:12.700076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:12.752218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d63686-2a9c-4996-a331-ccc22747a115 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:00:12.753532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d63686-2a9c-4996-a331-ccc22747a115 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:12.754051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d63686-2a9c-4996-a331-ccc22747a115 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:12.754618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d63686-2a9c-4996-a331-ccc22747a115 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:12.757160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-36d63686-2a9c-4996-a331-ccc22747a115 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:12.871275 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d63686-2a9c-4996-a331-ccc22747a115 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:00:12.873594 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d63686-2a9c-4996-a331-ccc22747a115 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:00:12.894289 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 685/1360] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:00:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:00:15.181560 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff5031-fd9f-4e4e-b91f-f31e7909119c None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:15.182414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff5031-fd9f-4e4e-b91f-f31e7909119c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:15.182699 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff5031-fd9f-4e4e-b91f-f31e7909119c None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:15.184288 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 676/1361] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:00:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:15.190025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af1b9a-7a17-4929-b221-35616cd9a03c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:15.190904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af1b9a-7a17-4929-b221-35616cd9a03c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:15.191534 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af1b9a-7a17-4929-b221-35616cd9a03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:15.269066 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-80af1b9a-7a17-4929-b221-35616cd9a03c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:15.270733 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-80af1b9a-7a17-4929-b221-35616cd9a03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:15.319857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:15.392830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 686/1362] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:00:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:15.439014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff5031-fd9f-4e4e-b91f-f31e7909119c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:00:15.440871 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-29ff5031-fd9f-4e4e-b91f-f31e7909119c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:15.528115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:15.656218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff5031-fd9f-4e4e-b91f-f31e7909119c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:00:15.804621 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c36f49-9ab4-4b16-8d61-819a2ec80487 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:00:15.805283 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c36f49-9ab4-4b16-8d61-819a2ec80487 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:15.805571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c36f49-9ab4-4b16-8d61-819a2ec80487 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:15.805853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c36f49-9ab4-4b16-8d61-819a2ec80487 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:15.807567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-28c36f49-9ab4-4b16-8d61-819a2ec80487 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:15.832813 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:15.912267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c36f49-9ab4-4b16-8d61-819a2ec80487 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:00:15.914028 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c36f49-9ab4-4b16-8d61-819a2ec80487 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:00:15.935949 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 677/1363] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:00:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:00:18.491838 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3e40a-fa66-4705-bd6b-d4b4ddd4ecfc None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:18.492369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3e40a-fa66-4705-bd6b-d4b4ddd4ecfc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:18.492488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3e40a-fa66-4705-bd6b-d4b4ddd4ecfc None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:18.493809 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 687/1364] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:18.500054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895e961-b407-4901-a83c-41a43d217fd9 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:18.500520 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895e961-b407-4901-a83c-41a43d217fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:18.500714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895e961-b407-4901-a83c-41a43d217fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:18.587054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7895e961-b407-4901-a83c-41a43d217fd9 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:18.588352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7895e961-b407-4901-a83c-41a43d217fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:18.698270 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:18.710556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 678/1365] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:00:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:18.750562 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340876ed-f54a-4612-a499-577b79c9568e None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:18.750939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340876ed-f54a-4612-a499-577b79c9568e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:18.751346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340876ed-f54a-4612-a499-577b79c9568e None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:18.752949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 688/1366] 10.222.0.69 () {54 vars in 961 bytes} [Thu Jul 14 19:00:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:18.759014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41afb39-cb00-4857-bf2d-82a372cf988d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:18.759332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41afb39-cb00-4857-bf2d-82a372cf988d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:18.759616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41afb39-cb00-4857-bf2d-82a372cf988d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:18.845345 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a41afb39-cb00-4857-bf2d-82a372cf988d None None] MFA Rules not processed for user `8d27d6df5cb64697adebd961dfbec324`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:18.846884 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a41afb39-cb00-4857-bf2d-82a372cf988d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:18.988293 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 679/1367] 10.222.0.69 () {58 vars in 1058 bytes} [Thu Jul 14 19:00:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:18.997558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340876ed-f54a-4612-a499-577b79c9568e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:00:19.001303 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-340876ed-f54a-4612-a499-577b79c9568e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:19.178907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340876ed-f54a-4612-a499-577b79c9568e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:00:19.202193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:19.309565 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ecac47-a07a-4089-b132-f24605acee6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:00:19.310191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ecac47-a07a-4089-b132-f24605acee6b service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:19.310473 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ecac47-a07a-4089-b132-f24605acee6b service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:19.310745 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ecac47-a07a-4089-b132-f24605acee6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:19.312419 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-09ecac47-a07a-4089-b132-f24605acee6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:19.413711 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ecac47-a07a-4089-b132-f24605acee6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:00:19.415556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ecac47-a07a-4089-b132-f24605acee6b service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:00:19.436180 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 689/1368] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:00:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:00:20.310278 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b3b76-bcca-4d20-aa6f-4f288d234a8e None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:20.310782 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b3b76-bcca-4d20-aa6f-4f288d234a8e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:20.310782 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b3b76-bcca-4d20-aa6f-4f288d234a8e None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:20.312411 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 680/1369] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:00:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:20.322331 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db53cb-6992-4500-ab7a-a27d9221d36a None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:20.322811 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db53cb-6992-4500-ab7a-a27d9221d36a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:20.322911 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db53cb-6992-4500-ab7a-a27d9221d36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:20.357786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:20.400332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-47db53cb-6992-4500-ab7a-a27d9221d36a None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:20.401737 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-47db53cb-6992-4500-ab7a-a27d9221d36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:20.517199 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 690/1370] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:20.861148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534cd8b-6edd-42c1-b094-942a62cb3e2e None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:20.861148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534cd8b-6edd-42c1-b094-942a62cb3e2e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:20.861148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534cd8b-6edd-42c1-b094-942a62cb3e2e None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:20.861148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 681/1371] 10.222.0.69 () {54 vars in 963 bytes} [Thu Jul 14 19:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:20.866349 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af08094-1877-4b04-a2e4-f1f2e0c7bf4d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:20.866349 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af08094-1877-4b04-a2e4-f1f2e0c7bf4d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:20.866710 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af08094-1877-4b04-a2e4-f1f2e0c7bf4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:20.947312 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9af08094-1877-4b04-a2e4-f1f2e0c7bf4d None None] MFA Rules not processed for user `31b09d72314349c6be0f906484895608`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:20.948666 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9af08094-1877-4b04-a2e4-f1f2e0c7bf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:21.007019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:21.035128 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 691/1372] 10.222.0.69 () {58 vars in 1060 bytes} [Thu Jul 14 19:00:20 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:21.042906 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534cd8b-6edd-42c1-b094-942a62cb3e2e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:00:21.045144 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8534cd8b-6edd-42c1-b094-942a62cb3e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:21.099963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:21.157017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534cd8b-6edd-42c1-b094-942a62cb3e2e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:00:21.235387 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1798fd-174a-4bec-a80e-d16804ad63e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:00:21.236048 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1798fd-174a-4bec-a80e-d16804ad63e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:21.236337 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1798fd-174a-4bec-a80e-d16804ad63e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:21.236613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1798fd-174a-4bec-a80e-d16804ad63e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:21.238257 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5f1798fd-174a-4bec-a80e-d16804ad63e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:21.353057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:21.368427 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1798fd-174a-4bec-a80e-d16804ad63e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:00:21.369895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1798fd-174a-4bec-a80e-d16804ad63e0 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:00:21.390771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 682/1373] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:00:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:00:21.724392 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eba8d0-9894-4b22-a171-7d4fa735f7bd None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:21.725219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eba8d0-9894-4b22-a171-7d4fa735f7bd None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:21.725219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eba8d0-9894-4b22-a171-7d4fa735f7bd None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:21.731942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 692/1374] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:00:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:21.735314 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eceb80-54bf-4f8b-878d-7759cf910415 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:21.735673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eceb80-54bf-4f8b-878d-7759cf910415 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:21.736090 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eceb80-54bf-4f8b-878d-7759cf910415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:21.823895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-11eceb80-54bf-4f8b-878d-7759cf910415 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:21.825180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-11eceb80-54bf-4f8b-878d-7759cf910415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:21.924904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 683/1375] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:00:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:21.969043 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eba8d0-9894-4b22-a171-7d4fa735f7bd None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:00:21.970563 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e0eba8d0-9894-4b22-a171-7d4fa735f7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:22.126339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eba8d0-9894-4b22-a171-7d4fa735f7bd None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:00:22.181422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:22.248330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90fa1b6-4485-4a44-a722-e4ba55e844df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:00:22.248992 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fa1b6-4485-4a44-a722-e4ba55e844df service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:22.249290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fa1b6-4485-4a44-a722-e4ba55e844df service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:22.249575 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fa1b6-4485-4a44-a722-e4ba55e844df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:22.251280 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d90fa1b6-4485-4a44-a722-e4ba55e844df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:22.347940 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90fa1b6-4485-4a44-a722-e4ba55e844df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:00:22.349730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90fa1b6-4485-4a44-a722-e4ba55e844df service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:00:22.369971 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 693/1376] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:00:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:00:24.586509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d256472-117d-4244-a273-3d37e029c662 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:24.587042 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d256472-117d-4244-a273-3d37e029c662 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:24.587497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d256472-117d-4244-a273-3d37e029c662 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:24.588813 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 684/1377] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:24.594880 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffd98f-c3c9-481e-a562-a23b0f797d51 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:24.595364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffd98f-c3c9-481e-a562-a23b0f797d51 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:24.595499 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffd98f-c3c9-481e-a562-a23b0f797d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:24.675950 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d6ffd98f-c3c9-481e-a562-a23b0f797d51 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:24.677080 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d6ffd98f-c3c9-481e-a562-a23b0f797d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:24.685436 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:24.784813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 694/1378] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:00:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:24.830647 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d256472-117d-4244-a273-3d37e029c662 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:00:24.832336 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0d256472-117d-4244-a273-3d37e029c662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:25.037062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:25.154025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d256472-117d-4244-a273-3d37e029c662 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:00:25.285620 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343f440-3d05-4392-bef0-2adc450e50d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:00:25.286477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343f440-3d05-4392-bef0-2adc450e50d5 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:25.286862 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343f440-3d05-4392-bef0-2adc450e50d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:25.287274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343f440-3d05-4392-bef0-2adc450e50d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:25.289037 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d343f440-3d05-4392-bef0-2adc450e50d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:25.310468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:25.391461 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343f440-3d05-4392-bef0-2adc450e50d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:00:25.393069 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343f440-3d05-4392-bef0-2adc450e50d5 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:00:25.413847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 685/1379] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:00:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:00:27.658832 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9df6a-314c-44d3-8501-749a6ad8b130 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:27.659523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9df6a-314c-44d3-8501-749a6ad8b130 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:27.659523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9df6a-314c-44d3-8501-749a6ad8b130 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:27.660818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 695/1380] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:27.666171 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91903ac8-64f6-4e58-9837-671abbcbaf7d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:27.666599 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91903ac8-64f6-4e58-9837-671abbcbaf7d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:27.666878 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91903ac8-64f6-4e58-9837-671abbcbaf7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:27.740540 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-91903ac8-64f6-4e58-9837-671abbcbaf7d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:27.741882 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-91903ac8-64f6-4e58-9837-671abbcbaf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:27.870530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:27.878837 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 686/1381] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:27.924787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9df6a-314c-44d3-8501-749a6ad8b130 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:00:27.927135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-aff9df6a-314c-44d3-8501-749a6ad8b130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:28.091765 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:28.103131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9df6a-314c-44d3-8501-749a6ad8b130 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:00:28.241334 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e1479-9870-4fce-bf89-b1c1c29466d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:00:28.241901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e1479-9870-4fce-bf89-b1c1c29466d9 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:28.242185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e1479-9870-4fce-bf89-b1c1c29466d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:28.242462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e1479-9870-4fce-bf89-b1c1c29466d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:28.244121 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a23e1479-9870-4fce-bf89-b1c1c29466d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:28.338887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e1479-9870-4fce-bf89-b1c1c29466d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:00:28.341057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e1479-9870-4fce-bf89-b1c1c29466d9 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:00:28.351888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:28.361415 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 696/1382] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:00:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:00:30.603355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1f00b-c9ca-45d3-871f-475986d39582 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:30.603895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1f00b-c9ca-45d3-871f-475986d39582 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:30.604074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1f00b-c9ca-45d3-871f-475986d39582 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:30.606149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 687/1383] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:00:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:30.611282 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8b301-c2c6-495f-a28c-0f1c5d0762ee None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:30.611831 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8b301-c2c6-495f-a28c-0f1c5d0762ee None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:30.611831 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8b301-c2c6-495f-a28c-0f1c5d0762ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:30.678535 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2bf8b301-c2c6-495f-a28c-0f1c5d0762ee None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:30.679781 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2bf8b301-c2c6-495f-a28c-0f1c5d0762ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:30.814315 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 697/1384] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:00:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:00:30.870988 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1f00b-c9ca-45d3-871f-475986d39582 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:00:30.873115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0fa1f00b-c9ca-45d3-871f-475986d39582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:31.027492 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1f00b-c9ca-45d3-871f-475986d39582 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:00:31.120022 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:31.163019 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b141332-404d-4a75-b482-eca728ebdf67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:00:31.163823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b141332-404d-4a75-b482-eca728ebdf67 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:31.164105 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b141332-404d-4a75-b482-eca728ebdf67 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:31.164399 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b141332-404d-4a75-b482-eca728ebdf67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:31.166839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4b141332-404d-4a75-b482-eca728ebdf67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:31.266343 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b141332-404d-4a75-b482-eca728ebdf67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:00:31.268411 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b141332-404d-4a75-b482-eca728ebdf67 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:00:31.291535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 688/1385] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:00:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:00:33.929382 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17376447-288e-42a4-b20c-2c2971d0631b None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:33.929968 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17376447-288e-42a4-b20c-2c2971d0631b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:33.929968 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17376447-288e-42a4-b20c-2c2971d0631b None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:33.931391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 698/1386] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:33.937990 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a61088-9b25-4a5f-8d0d-3b3cbdc9b6b3 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:33.938459 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a61088-9b25-4a5f-8d0d-3b3cbdc9b6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:33.938734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a61088-9b25-4a5f-8d0d-3b3cbdc9b6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:34.017987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-49a61088-9b25-4a5f-8d0d-3b3cbdc9b6b3 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:34.019258 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-49a61088-9b25-4a5f-8d0d-3b3cbdc9b6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:34.055145 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:34.126939 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 689/1387] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:00:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:34.181873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17376447-288e-42a4-b20c-2c2971d0631b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:00:34.183431 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-17376447-288e-42a4-b20c-2c2971d0631b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:34.269383 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:34.343726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17376447-288e-42a4-b20c-2c2971d0631b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:00:34.461298 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be2fbd-ea02-4484-a5f0-26122536c6f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:00:34.461854 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be2fbd-ea02-4484-a5f0-26122536c6f7 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:34.462184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be2fbd-ea02-4484-a5f0-26122536c6f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:34.462402 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be2fbd-ea02-4484-a5f0-26122536c6f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:34.464080 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-04be2fbd-ea02-4484-a5f0-26122536c6f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:34.513010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:34.575771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be2fbd-ea02-4484-a5f0-26122536c6f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:00:34.577401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be2fbd-ea02-4484-a5f0-26122536c6f7 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:00:34.595770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 699/1388] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:00:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:00:36.756120 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8d27b-8448-4b58-ac11-a5c5da30a3f9 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:36.756688 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8d27b-8448-4b58-ac11-a5c5da30a3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:36.756688 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8d27b-8448-4b58-ac11-a5c5da30a3f9 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:36.758240 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 690/1389] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:36.763899 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624effa0-211a-4d51-b101-6a759329eb8b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:36.764348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624effa0-211a-4d51-b101-6a759329eb8b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:36.764455 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624effa0-211a-4d51-b101-6a759329eb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:36.833120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-624effa0-211a-4d51-b101-6a759329eb8b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:36.834357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-624effa0-211a-4d51-b101-6a759329eb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:36.893931 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d442f-6784-4f47-b1e1-8bb656473743 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:36.894212 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d442f-6784-4f47-b1e1-8bb656473743 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:36.894490 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d442f-6784-4f47-b1e1-8bb656473743 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:36.896114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 691/1390] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 19:00:36.901053 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463dd9f-332e-40da-8189-ef46a195a9e8 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:36.901342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463dd9f-332e-40da-8189-ef46a195a9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:36.901622 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463dd9f-332e-40da-8189-ef46a195a9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:36.933250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 700/1391] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:36.966565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-8463dd9f-332e-40da-8189-ef46a195a9e8 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:36.967964 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8463dd9f-332e-40da-8189-ef46a195a9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:36.977808 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624effa0-211a-4d51-b101-6a759329eb8b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:00:36.978911 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-624effa0-211a-4d51-b101-6a759329eb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:36.993200 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:37.070082 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 692/1392] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:00:37.102787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624effa0-211a-4d51-b101-6a759329eb8b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:00:37.126636 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60090f75-113e-4b9a-9005-3bb895476a28 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:37.126934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60090f75-113e-4b9a-9005-3bb895476a28 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:37.127246 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60090f75-113e-4b9a-9005-3bb895476a28 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:37.128634 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 693/1393] 10.222.0.69 () {54 vars in 963 bytes} [Thu Jul 14 19:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 19:00:37.133578 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe820ce4-bc81-4846-bb5f-66e962ccdaee None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:37.133890 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe820ce4-bc81-4846-bb5f-66e962ccdaee None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:37.134153 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe820ce4-bc81-4846-bb5f-66e962ccdaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:37.149245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:37.212381 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-fe820ce4-bc81-4846-bb5f-66e962ccdaee None None] MFA Rules not processed for user `e5d76eae62b54521a95f36284118fada`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:37.213645 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-fe820ce4-bc81-4846-bb5f-66e962ccdaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:37.223184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317dc8c-a905-43ae-b05a-727201227dc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:00:37.223858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317dc8c-a905-43ae-b05a-727201227dc3 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:37.224133 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317dc8c-a905-43ae-b05a-727201227dc3 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:37.224404 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317dc8c-a905-43ae-b05a-727201227dc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:37.226034 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3317dc8c-a905-43ae-b05a-727201227dc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:37.235303 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:37.291537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 694/1394] 10.222.0.69 () {58 vars in 1060 bytes} [Thu Jul 14 19:00:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:00:37.300560 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe820ce4-bc81-4846-bb5f-66e962ccdaee None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:00:37.308481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe820ce4-bc81-4846-bb5f-66e962ccdaee None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:00:37.318972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317dc8c-a905-43ae-b05a-727201227dc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:00:37.320673 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317dc8c-a905-43ae-b05a-727201227dc3 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:00:37.340697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 701/1395] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:00:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:00:37.384782 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112dabb2-f320-4721-90ee-9f029984aaea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5d76eae62b54521a95f36284118fada', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:00:37.385432 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dabb2-f320-4721-90ee-9f029984aaea service glance] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:37.385713 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dabb2-f320-4721-90ee-9f029984aaea service glance] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:37.385988 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dabb2-f320-4721-90ee-9f029984aaea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:37.387713 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-112dabb2-f320-4721-90ee-9f029984aaea service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:37.419174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:37.500524 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112dabb2-f320-4721-90ee-9f029984aaea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:00:37.501957 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112dabb2-f320-4721-90ee-9f029984aaea service glance] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:00:37.520391 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 695/1396] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:00:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:00:38.397690 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27692eb4-69f6-4dd5-a29f-1efd6cbf573a None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:38.398171 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27692eb4-69f6-4dd5-a29f-1efd6cbf573a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:38.398397 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27692eb4-69f6-4dd5-a29f-1efd6cbf573a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:38.463080 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-27692eb4-69f6-4dd5-a29f-1efd6cbf573a None None] MFA Rules not processed for user `4ef0fad9a71d43bfbaba9c87dfed41c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:38.464313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-27692eb4-69f6-4dd5-a29f-1efd6cbf573a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:38.555395 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:38.558763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 702/1397] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 14 19:00:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:38.601437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e1aa8-2b76-4c63-985f-e62529e3a598 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:38.601928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e1aa8-2b76-4c63-985f-e62529e3a598 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:38.602023 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e1aa8-2b76-4c63-985f-e62529e3a598 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:38.603449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 696/1398] 10.222.0.69 () {54 vars in 942 bytes} [Thu Jul 14 19:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:38.609916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad588a-fe0e-4ccb-9283-78d1b068c28f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:38.610273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad588a-fe0e-4ccb-9283-78d1b068c28f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:38.610559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad588a-fe0e-4ccb-9283-78d1b068c28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:38.688261 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-31ad588a-fe0e-4ccb-9283-78d1b068c28f None None] MFA Rules not processed for user `4368f68724f946eca53084a9ec7ff590`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:38.689383 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-31ad588a-fe0e-4ccb-9283-78d1b068c28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:38.797230 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 703/1399] 10.222.0.69 () {58 vars in 1039 bytes} [Thu Jul 14 19:00:38 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:38.806609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e1aa8-2b76-4c63-985f-e62529e3a598 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:00:38.808138 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f95e1aa8-2b76-4c63-985f-e62529e3a598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:38.941768 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:38.944320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e1aa8-2b76-4c63-985f-e62529e3a598 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:00:39.052167 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeccacdb-8853-4096-9638-0cab959e7cba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4368f68724f946eca53084a9ec7ff590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:00:39.052810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccacdb-8853-4096-9638-0cab959e7cba service swift] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:39.053087 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccacdb-8853-4096-9638-0cab959e7cba service swift] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:39.053353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccacdb-8853-4096-9638-0cab959e7cba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:39.055004 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-aeccacdb-8853-4096-9638-0cab959e7cba service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:39.137031 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeccacdb-8853-4096-9638-0cab959e7cba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:00:39.139134 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeccacdb-8853-4096-9638-0cab959e7cba service swift] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:00:39.147787 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 697/1400] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 14 19:00:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 19:00:39.550663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733538f1-9e6e-4773-b694-64ff6806bba5 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:39.551242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733538f1-9e6e-4773-b694-64ff6806bba5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:39.551242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733538f1-9e6e-4773-b694-64ff6806bba5 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:39.552702 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 704/1401] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:39.557960 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6f742-52c7-4276-9068-0226792dba50 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:39.558470 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6f742-52c7-4276-9068-0226792dba50 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:39.559683 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6f742-52c7-4276-9068-0226792dba50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:39.596801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:39.627786 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-aee6f742-52c7-4276-9068-0226792dba50 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:39.629558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-aee6f742-52c7-4276-9068-0226792dba50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:39.733860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 698/1402] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:00:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:00:39.779669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733538f1-9e6e-4773-b694-64ff6806bba5 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:00:39.781301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-733538f1-9e6e-4773-b694-64ff6806bba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:39.857960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:39.963038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733538f1-9e6e-4773-b694-64ff6806bba5 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:00:40.094532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965e7a1-9ab6-41c2-9ec6-68268e5b2997 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:00:40.095202 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965e7a1-9ab6-41c2-9ec6-68268e5b2997 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:40.095498 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965e7a1-9ab6-41c2-9ec6-68268e5b2997 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:40.095776 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965e7a1-9ab6-41c2-9ec6-68268e5b2997 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:40.098566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c965e7a1-9ab6-41c2-9ec6-68268e5b2997 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:40.118564 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:40.199927 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965e7a1-9ab6-41c2-9ec6-68268e5b2997 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:00:40.201556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965e7a1-9ab6-41c2-9ec6-68268e5b2997 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:00:40.223359 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 705/1403] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:00:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:00:42.447888 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247daa45-e0cc-4da1-8629-afeb17a53ba0 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:42.448636 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247daa45-e0cc-4da1-8629-afeb17a53ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:42.448636 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247daa45-e0cc-4da1-8629-afeb17a53ba0 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:42.455759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd81be-4c8e-4e2a-a3f7-49c0234117e4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:42.456308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd81be-4c8e-4e2a-a3f7-49c0234117e4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:42.456433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd81be-4c8e-4e2a-a3f7-49c0234117e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:42.460413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 699/1404] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:00:42 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:42.545476 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-35fd81be-4c8e-4e2a-a3f7-49c0234117e4 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:42.546783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-35fd81be-4c8e-4e2a-a3f7-49c0234117e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:42.666179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:42.675258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 706/1405] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:42.744997 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247daa45-e0cc-4da1-8629-afeb17a53ba0 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:00:42.744997 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-247daa45-e0cc-4da1-8629-afeb17a53ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:42.857799 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:42.905831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247daa45-e0cc-4da1-8629-afeb17a53ba0 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:00:43.050317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3fd93-acab-4a34-9dce-8ef943f92832 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:00:43.050974 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3fd93-acab-4a34-9dce-8ef943f92832 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:43.051286 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3fd93-acab-4a34-9dce-8ef943f92832 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:43.051574 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3fd93-acab-4a34-9dce-8ef943f92832 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:43.053210 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-06c3fd93-acab-4a34-9dce-8ef943f92832 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:43.171255 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3fd93-acab-4a34-9dce-8ef943f92832 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:00:43.173675 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3fd93-acab-4a34-9dce-8ef943f92832 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:00:43.200797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 700/1406] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:00:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:00:43.361266 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:45.393740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9167b64-25a5-483b-bbf9-9f3722c029dd None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:45.394310 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9167b64-25a5-483b-bbf9-9f3722c029dd None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:45.394402 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9167b64-25a5-483b-bbf9-9f3722c029dd None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:45.396737 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 707/1407] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:00:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:00:45.402237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfef18-1ded-48f8-8137-a46ac2067de0 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:45.403407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfef18-1ded-48f8-8137-a46ac2067de0 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:45.403521 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfef18-1ded-48f8-8137-a46ac2067de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:45.475791 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-e4bfef18-1ded-48f8-8137-a46ac2067de0 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:00:45.477105 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e4bfef18-1ded-48f8-8137-a46ac2067de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:45.585341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 701/1408] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:00:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:00:45.630927 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9167b64-25a5-483b-bbf9-9f3722c029dd None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:00:45.632490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e9167b64-25a5-483b-bbf9-9f3722c029dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:45.800699 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9167b64-25a5-483b-bbf9-9f3722c029dd None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:00:45.880720 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:45.930908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d274c74-4ef9-45d3-ba6f-8224cd65b185 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:00:45.931597 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d274c74-4ef9-45d3-ba6f-8224cd65b185 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:00:45.931895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d274c74-4ef9-45d3-ba6f-8224cd65b185 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:00:45.932170 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d274c74-4ef9-45d3-ba6f-8224cd65b185 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:00:45.934526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1d274c74-4ef9-45d3-ba6f-8224cd65b185 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:00:46.042746 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d274c74-4ef9-45d3-ba6f-8224cd65b185 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:00:46.045154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d274c74-4ef9-45d3-ba6f-8224cd65b185 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:00:46.067151 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 708/1409] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:00:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:00:55.583575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=79 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:00:56.064689 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=64 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:02.861421 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241df8a9-7c0c-44a7-8175-3c2343b8dc71 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:02.862002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241df8a9-7c0c-44a7-8175-3c2343b8dc71 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:02.862002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241df8a9-7c0c-44a7-8175-3c2343b8dc71 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:02.863404 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 702/1410] 10.222.0.69 () {54 vars in 918 bytes} [Thu Jul 14 19:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:02.868810 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85bc475-1983-44e7-9313-4fdc868ac0f6 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:02.869191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85bc475-1983-44e7-9313-4fdc868ac0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:02.869404 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85bc475-1983-44e7-9313-4fdc868ac0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:02.943249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d85bc475-1983-44e7-9313-4fdc868ac0f6 None None] MFA Rules not processed for user `c577fa8670e34449a80de0c8e0a15a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:02.944453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d85bc475-1983-44e7-9313-4fdc868ac0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:03.050145 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 709/1411] 10.222.0.69 () {58 vars in 1015 bytes} [Thu Jul 14 19:01:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:01:03.064899 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a62750-4746-467c-85a7-ecea9e7e90cc None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:03.065193 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a62750-4746-467c-85a7-ecea9e7e90cc None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:03.065471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a62750-4746-467c-85a7-ecea9e7e90cc None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:03.066820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 703/1412] 10.222.0.69 () {54 vars in 918 bytes} [Thu Jul 14 19:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:03.071528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59405e-be68-48af-9552-06eca7f86d57 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:03.071848 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59405e-be68-48af-9552-06eca7f86d57 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:03.072139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59405e-be68-48af-9552-06eca7f86d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:03.143804 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:03.147014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-cb59405e-be68-48af-9552-06eca7f86d57 None None] MFA Rules not processed for user `8d27d6df5cb64697adebd961dfbec324`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:03.148273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cb59405e-be68-48af-9552-06eca7f86d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:03.265775 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 710/1413] 10.222.0.69 () {58 vars in 1015 bytes} [Thu Jul 14 19:01:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:05.778563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777e943-693a-4619-84e1-b96efb11c7b6 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:05.779132 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777e943-693a-4619-84e1-b96efb11c7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:05.779132 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777e943-693a-4619-84e1-b96efb11c7b6 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:05.780494 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 19:01:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:05.785841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fafd8-7363-4c42-9bbb-1524dc3c4ae3 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:05.786239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fafd8-7363-4c42-9bbb-1524dc3c4ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:05.786402 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fafd8-7363-4c42-9bbb-1524dc3c4ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:05.856036 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6e9fafd8-7363-4c42-9bbb-1524dc3c4ae3 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:05.857259 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6e9fafd8-7363-4c42-9bbb-1524dc3c4ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:05.912438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:05.954272 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 19:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:05.997382 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777e943-693a-4619-84e1-b96efb11c7b6 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:05.998887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8777e943-693a-4619-84e1-b96efb11c7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:06.142473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777e943-693a-4619-84e1-b96efb11c7b6 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:06.214824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:06.260479 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b69b12-38bb-4f7f-a9be-ab63e146e8aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:06.261445 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b69b12-38bb-4f7f-a9be-ab63e146e8aa service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:06.262082 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b69b12-38bb-4f7f-a9be-ab63e146e8aa service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:06.262377 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b69b12-38bb-4f7f-a9be-ab63e146e8aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:06.264076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e6b69b12-38bb-4f7f-a9be-ab63e146e8aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:06.352173 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b69b12-38bb-4f7f-a9be-ab63e146e8aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:06.353758 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b69b12-38bb-4f7f-a9be-ab63e146e8aa service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:06.372535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 705/1416] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:07.270143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b00a9-80e9-4c16-94f4-644c748b6e1d None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:07.270667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b00a9-80e9-4c16-94f4-644c748b6e1d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:07.270667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b00a9-80e9-4c16-94f4-644c748b6e1d None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:07.272314 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 19:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:07.277864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ef5c3-0e4f-46d5-a1cc-cbb5e557ae97 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:07.278291 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ef5c3-0e4f-46d5-a1cc-cbb5e557ae97 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:07.278404 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ef5c3-0e4f-46d5-a1cc-cbb5e557ae97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:07.352984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b74ef5c3-0e4f-46d5-a1cc-cbb5e557ae97 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:07.354205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b74ef5c3-0e4f-46d5-a1cc-cbb5e557ae97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:07.381057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:07.450780 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 19:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:07.599960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b00a9-80e9-4c16-94f4-644c748b6e1d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:07.601471 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-633b00a9-80e9-4c16-94f4-644c748b6e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:07.745803 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b00a9-80e9-4c16-94f4-644c748b6e1d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:07.784884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:07.857471 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851dbf6-b1a3-4e95-bff0-c6d9207b69ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:07.858041 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851dbf6-b1a3-4e95-bff0-c6d9207b69ae service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:07.858330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851dbf6-b1a3-4e95-bff0-c6d9207b69ae service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:07.858605 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851dbf6-b1a3-4e95-bff0-c6d9207b69ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:07.860266 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1851dbf6-b1a3-4e95-bff0-c6d9207b69ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:07.950837 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851dbf6-b1a3-4e95-bff0-c6d9207b69ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:07.952497 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851dbf6-b1a3-4e95-bff0-c6d9207b69ae service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:07.970275 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 713/1419] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:08.848437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fdd20-0627-469d-b645-3348ca75c21b None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:08.849105 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fdd20-0627-469d-b645-3348ca75c21b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:08.849105 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fdd20-0627-469d-b645-3348ca75c21b None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:08.850396 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 19:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:08.855432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc5ab4-202e-4f4d-8437-70f5a33f26fb None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:08.855858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc5ab4-202e-4f4d-8437-70f5a33f26fb None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:08.856014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc5ab4-202e-4f4d-8437-70f5a33f26fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:08.909647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:08.931087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-f5cc5ab4-202e-4f4d-8437-70f5a33f26fb None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:08.932343 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f5cc5ab4-202e-4f4d-8437-70f5a33f26fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:09.029767 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 19:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:09.179353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fdd20-0627-469d-b645-3348ca75c21b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:09.180801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-486fdd20-0627-469d-b645-3348ca75c21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:09.327006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fdd20-0627-469d-b645-3348ca75c21b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:09.338922 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:09.439917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575acd12-a771-423a-970f-4e73f2deebc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:09.440627 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575acd12-a771-423a-970f-4e73f2deebc4 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:09.440918 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575acd12-a771-423a-970f-4e73f2deebc4 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:09.454628 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575acd12-a771-423a-970f-4e73f2deebc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:09.454628 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-575acd12-a771-423a-970f-4e73f2deebc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:09.531681 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575acd12-a771-423a-970f-4e73f2deebc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:09.533284 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575acd12-a771-423a-970f-4e73f2deebc4 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:09.551717 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 708/1422] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:10.409363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3b541-94f9-401e-9caf-0f657e20ca1a None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:10.409906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3b541-94f9-401e-9caf-0f657e20ca1a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:10.409906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3b541-94f9-401e-9caf-0f657e20ca1a None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:10.411361 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 19:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:10.416413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a4c9e-9bd8-4ee2-9c76-ee8fd4584cbd None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:10.416826 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a4c9e-9bd8-4ee2-9c76-ee8fd4584cbd None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:10.417042 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a4c9e-9bd8-4ee2-9c76-ee8fd4584cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:10.435673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:10.490267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-234a4c9e-9bd8-4ee2-9c76-ee8fd4584cbd None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:10.491496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-234a4c9e-9bd8-4ee2-9c76-ee8fd4584cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:10.586914 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 19:01:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:10.736772 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3b541-94f9-401e-9caf-0f657e20ca1a None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:10.738201 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-64b3b541-94f9-401e-9caf-0f657e20ca1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:10.846980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:10.882025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3b541-94f9-401e-9caf-0f657e20ca1a None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:11.006665 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fc799-a8ee-4a37-acdb-ac4b5605a0e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:11.007657 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fc799-a8ee-4a37-acdb-ac4b5605a0e2 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:11.008078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fc799-a8ee-4a37-acdb-ac4b5605a0e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:11.008485 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fc799-a8ee-4a37-acdb-ac4b5605a0e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:11.011670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-908fc799-a8ee-4a37-acdb-ac4b5605a0e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:11.098709 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:11.113001 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fc799-a8ee-4a37-acdb-ac4b5605a0e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:11.114679 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fc799-a8ee-4a37-acdb-ac4b5605a0e2 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:11.133111 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 716/1425] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:12.002689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1ff55-b05c-4182-9c4f-19ff4d1f64f6 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:12.003264 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1ff55-b05c-4182-9c4f-19ff4d1f64f6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:12.003264 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1ff55-b05c-4182-9c4f-19ff4d1f64f6 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:12.004876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 19:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:12.010364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f1954-e696-4bc2-a31b-f020036ee980 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:12.010822 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f1954-e696-4bc2-a31b-f020036ee980 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:12.010956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f1954-e696-4bc2-a31b-f020036ee980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:12.085302 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-388f1954-e696-4bc2-a31b-f020036ee980 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:12.086536 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-388f1954-e696-4bc2-a31b-f020036ee980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:12.182699 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 19:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:12.335966 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1ff55-b05c-4182-9c4f-19ff4d1f64f6 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:12.337502 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-43a1ff55-b05c-4182-9c4f-19ff4d1f64f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:12.421608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:12.483777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1ff55-b05c-4182-9c4f-19ff4d1f64f6 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:12.601632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64f84d-6af3-4f16-9523-eb4640713ff5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:12.602201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64f84d-6af3-4f16-9523-eb4640713ff5 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:12.602486 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64f84d-6af3-4f16-9523-eb4640713ff5 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:12.602769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64f84d-6af3-4f16-9523-eb4640713ff5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:12.604505 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7d64f84d-6af3-4f16-9523-eb4640713ff5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:12.658329 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:12.700088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64f84d-6af3-4f16-9523-eb4640713ff5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:12.701703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64f84d-6af3-4f16-9523-eb4640713ff5 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:12.720811 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 711/1428] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:13.591254 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1567f7e-0e89-4eb4-ba5e-3bffb9b5c27d None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:13.591815 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1567f7e-0e89-4eb4-ba5e-3bffb9b5c27d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:13.591815 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1567f7e-0e89-4eb4-ba5e-3bffb9b5c27d None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:13.593228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 19:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:13.598316 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b08e-9b52-453d-9de3-4703630805bd None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:13.598760 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b08e-9b52-453d-9de3-4703630805bd None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:13.598870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b08e-9b52-453d-9de3-4703630805bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:13.672857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ede3b08e-9b52-453d-9de3-4703630805bd None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:13.674070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ede3b08e-9b52-453d-9de3-4703630805bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:13.775403 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 19:01:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:13.926349 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1567f7e-0e89-4eb4-ba5e-3bffb9b5c27d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:13.927976 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e1567f7e-0e89-4eb4-ba5e-3bffb9b5c27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:13.976857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:14.078503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1567f7e-0e89-4eb4-ba5e-3bffb9b5c27d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:14.189753 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be974dd2-1e03-4631-95e9-75ed0544fe10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:14.190305 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be974dd2-1e03-4631-95e9-75ed0544fe10 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:14.190588 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be974dd2-1e03-4631-95e9-75ed0544fe10 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:14.190862 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be974dd2-1e03-4631-95e9-75ed0544fe10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:14.192526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-be974dd2-1e03-4631-95e9-75ed0544fe10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:14.199332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:14.282733 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be974dd2-1e03-4631-95e9-75ed0544fe10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:14.284401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be974dd2-1e03-4631-95e9-75ed0544fe10 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:14.305641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 719/1431] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:15.208321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0da4ed-321b-49fb-b048-7aac579a33a8 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:15.208862 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0da4ed-321b-49fb-b048-7aac579a33a8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:15.208862 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0da4ed-321b-49fb-b048-7aac579a33a8 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:15.210167 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 19:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:15.215438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14aa5b-f39b-4ca5-8183-2c0e0c997a1d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:15.215855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14aa5b-f39b-4ca5-8183-2c0e0c997a1d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:15.216012 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14aa5b-f39b-4ca5-8183-2c0e0c997a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:15.291662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-cf14aa5b-f39b-4ca5-8183-2c0e0c997a1d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:15.293395 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cf14aa5b-f39b-4ca5-8183-2c0e0c997a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:15.361598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:15.391141 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 19:01:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:15.549095 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0da4ed-321b-49fb-b048-7aac579a33a8 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:15.550862 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-fc0da4ed-321b-49fb-b048-7aac579a33a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:15.568882 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:15.705868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0da4ed-321b-49fb-b048-7aac579a33a8 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:15.793659 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:15.819261 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664d66d-4c65-4f2e-a26b-c6ab32f3c72d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:15.819861 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664d66d-4c65-4f2e-a26b-c6ab32f3c72d service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:15.820149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664d66d-4c65-4f2e-a26b-c6ab32f3c72d service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:15.820425 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664d66d-4c65-4f2e-a26b-c6ab32f3c72d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:15.822053 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a664d66d-4c65-4f2e-a26b-c6ab32f3c72d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:15.911463 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a664d66d-4c65-4f2e-a26b-c6ab32f3c72d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:15.913068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a664d66d-4c65-4f2e-a26b-c6ab32f3c72d service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:15.930777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 714/1434] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:16.791051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83862203-f0cd-4acf-a3de-3cc3efa790d4 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:16.791648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83862203-f0cd-4acf-a3de-3cc3efa790d4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:16.791648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83862203-f0cd-4acf-a3de-3cc3efa790d4 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:16.793561 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 19:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:16.798502 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677e212-9968-4f8d-8064-f13c48d5d393 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:16.798951 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677e212-9968-4f8d-8064-f13c48d5d393 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:16.799061 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677e212-9968-4f8d-8064-f13c48d5d393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:16.875128 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-2677e212-9968-4f8d-8064-f13c48d5d393 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:16.876343 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2677e212-9968-4f8d-8064-f13c48d5d393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:16.923652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:16.978723 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 19:01:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:17.129199 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83862203-f0cd-4acf-a3de-3cc3efa790d4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:17.130723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-83862203-f0cd-4acf-a3de-3cc3efa790d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:17.283392 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83862203-f0cd-4acf-a3de-3cc3efa790d4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:17.338152 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:17.396543 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16710054-8245-4884-a1bc-04746a7950ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:17.397125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16710054-8245-4884-a1bc-04746a7950ae service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:17.397410 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16710054-8245-4884-a1bc-04746a7950ae service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:17.397683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16710054-8245-4884-a1bc-04746a7950ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:17.399332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-16710054-8245-4884-a1bc-04746a7950ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:17.486724 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16710054-8245-4884-a1bc-04746a7950ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:17.488433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16710054-8245-4884-a1bc-04746a7950ae service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:17.506628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 722/1437] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:18.378556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977071a-0a06-4aa8-80b0-cdd6c456b923 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:18.379146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977071a-0a06-4aa8-80b0-cdd6c456b923 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:18.379146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977071a-0a06-4aa8-80b0-cdd6c456b923 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:18.380516 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 19:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:18.386065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35a63a-3209-40a5-ac83-b1858b74b3f5 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:18.386481 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35a63a-3209-40a5-ac83-b1858b74b3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:18.386632 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35a63a-3209-40a5-ac83-b1858b74b3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:18.459420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-4b35a63a-3209-40a5-ac83-b1858b74b3f5 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:18.460625 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4b35a63a-3209-40a5-ac83-b1858b74b3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:18.462635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:18.556281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 19:01:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:18.706967 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1977071a-0a06-4aa8-80b0-cdd6c456b923 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:18.708462 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1977071a-0a06-4aa8-80b0-cdd6c456b923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:18.851849 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1977071a-0a06-4aa8-80b0-cdd6c456b923 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:18.878701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:18.965716 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9104f7-b4ef-4075-95c7-75e0a9f746ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:18.966295 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9104f7-b4ef-4075-95c7-75e0a9f746ef service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:18.966581 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9104f7-b4ef-4075-95c7-75e0a9f746ef service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:18.966857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9104f7-b4ef-4075-95c7-75e0a9f746ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:18.968511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-cf9104f7-b4ef-4075-95c7-75e0a9f746ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:19.058922 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9104f7-b4ef-4075-95c7-75e0a9f746ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:19.060546 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9104f7-b4ef-4075-95c7-75e0a9f746ef service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:19.078125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 717/1440] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:20.993007 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccfb44-51fd-4b78-9083-eb38c875042c None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:20.993520 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccfb44-51fd-4b78-9083-eb38c875042c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:20.993628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccfb44-51fd-4b78-9083-eb38c875042c None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:20.995571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 19:01:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:21.001603 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ad67c-c5f3-496a-a75b-14d3b84e01a7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:21.002069 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ad67c-c5f3-496a-a75b-14d3b84e01a7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:21.002171 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ad67c-c5f3-496a-a75b-14d3b84e01a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:21.043714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:21.068753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-8b9ad67c-c5f3-496a-a75b-14d3b84e01a7 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:21.069972 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8b9ad67c-c5f3-496a-a75b-14d3b84e01a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:21.166686 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 19:01:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:21.205965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ccfb44-51fd-4b78-9083-eb38c875042c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:21.207545 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-38ccfb44-51fd-4b78-9083-eb38c875042c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:21.315258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ccfb44-51fd-4b78-9083-eb38c875042c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:21.346513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:21.390040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb396f-5f2b-4f59-a659-866fea47a478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:21.390682 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb396f-5f2b-4f59-a659-866fea47a478 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:21.390962 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb396f-5f2b-4f59-a659-866fea47a478 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:21.391268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb396f-5f2b-4f59-a659-866fea47a478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:21.392878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-65eb396f-5f2b-4f59-a659-866fea47a478 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:21.619238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb396f-5f2b-4f59-a659-866fea47a478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:21.620749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb396f-5f2b-4f59-a659-866fea47a478 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:21.638714 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 725/1443] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:23.809751 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0b8f4-b517-4a61-a260-96ee15483864 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:23.810346 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0b8f4-b517-4a61-a260-96ee15483864 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:23.810346 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0b8f4-b517-4a61-a260-96ee15483864 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:23.811757 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 19:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:23.817537 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641f2f3-2451-4cae-9eb9-40798ab2c3f6 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:23.817962 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641f2f3-2451-4cae-9eb9-40798ab2c3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:23.818100 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641f2f3-2451-4cae-9eb9-40798ab2c3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:23.884592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e641f2f3-2451-4cae-9eb9-40798ab2c3f6 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:23.886442 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e641f2f3-2451-4cae-9eb9-40798ab2c3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:23.899374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:23.982610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 19:01:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:24.021588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0b8f4-b517-4a61-a260-96ee15483864 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:24.023129 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1fa0b8f4-b517-4a61-a260-96ee15483864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:24.124364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0b8f4-b517-4a61-a260-96ee15483864 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:24.136849 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:24.197199 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f68cee-3f27-412d-a054-f410c15d3f5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:24.198391 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f68cee-3f27-412d-a054-f410c15d3f5d service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:24.198675 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f68cee-3f27-412d-a054-f410c15d3f5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:24.198952 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f68cee-3f27-412d-a054-f410c15d3f5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:24.200604 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-08f68cee-3f27-412d-a054-f410c15d3f5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:24.306655 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f68cee-3f27-412d-a054-f410c15d3f5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:24.308717 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f68cee-3f27-412d-a054-f410c15d3f5d service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:24.326355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 720/1446] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:26.756589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc39360-2fbe-4a35-9181-76dbbd35c482 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:26.757169 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc39360-2fbe-4a35-9181-76dbbd35c482 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:26.757169 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc39360-2fbe-4a35-9181-76dbbd35c482 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:26.758561 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 19:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:26.765953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7144ad5-3aed-494b-9267-3fb9263c98a9 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:26.767331 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7144ad5-3aed-494b-9267-3fb9263c98a9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:26.767331 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7144ad5-3aed-494b-9267-3fb9263c98a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:26.814392 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:26.833017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f7144ad5-3aed-494b-9267-3fb9263c98a9 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:26.835292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f7144ad5-3aed-494b-9267-3fb9263c98a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:26.930617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 19:01:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:26.970660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc39360-2fbe-4a35-9181-76dbbd35c482 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:26.972266 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ffc39360-2fbe-4a35-9181-76dbbd35c482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:27.079929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc39360-2fbe-4a35-9181-76dbbd35c482 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:27.126483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:27.160758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de5e91-afa4-4014-9b22-7ff1b058a656 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:27.161404 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de5e91-afa4-4014-9b22-7ff1b058a656 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:27.161686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de5e91-afa4-4014-9b22-7ff1b058a656 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:27.161962 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de5e91-afa4-4014-9b22-7ff1b058a656 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:27.163615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-38de5e91-afa4-4014-9b22-7ff1b058a656 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:27.270784 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de5e91-afa4-4014-9b22-7ff1b058a656 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:27.272521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de5e91-afa4-4014-9b22-7ff1b058a656 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:27.290960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 728/1449] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:29.433643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4cf361-d70c-4186-939e-60e9c74195bb None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:29.434188 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4cf361-d70c-4186-939e-60e9c74195bb None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:29.434302 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4cf361-d70c-4186-939e-60e9c74195bb None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:29.436381 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 19:01:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:29.441752 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb7d93-0c02-430d-8e63-b67c326fb6b5 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:29.442181 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb7d93-0c02-430d-8e63-b67c326fb6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:29.442316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb7d93-0c02-430d-8e63-b67c326fb6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:29.505903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-58eb7d93-0c02-430d-8e63-b67c326fb6b5 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:29.507108 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-58eb7d93-0c02-430d-8e63-b67c326fb6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:29.527216 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:29.600502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 19:01:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:29.640663 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4cf361-d70c-4186-939e-60e9c74195bb None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:29.642319 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2e4cf361-d70c-4186-939e-60e9c74195bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:29.747850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4cf361-d70c-4186-939e-60e9c74195bb None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:29.766975 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:29.819635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a1f8b-6053-4415-95de-c411f0d6135d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:29.820274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a1f8b-6053-4415-95de-c411f0d6135d service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:29.820555 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a1f8b-6053-4415-95de-c411f0d6135d service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:29.820830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a1f8b-6053-4415-95de-c411f0d6135d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:29.822466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-955a1f8b-6053-4415-95de-c411f0d6135d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:29.930513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a1f8b-6053-4415-95de-c411f0d6135d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:29.931975 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a1f8b-6053-4415-95de-c411f0d6135d service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:29.949675 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 723/1452] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:32.020035 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f920d16-d5d9-4e2f-903b-9f4cffd90615 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:32.020553 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f920d16-d5d9-4e2f-903b-9f4cffd90615 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:32.020670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f920d16-d5d9-4e2f-903b-9f4cffd90615 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:32.022249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 19:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:32.028304 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd841c-791c-4a7c-b0d1-1cd4419cf2e5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:32.028710 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd841c-791c-4a7c-b0d1-1cd4419cf2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:32.028867 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd841c-791c-4a7c-b0d1-1cd4419cf2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:32.091242 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:32.094102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-64bd841c-791c-4a7c-b0d1-1cd4419cf2e5 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:32.095369 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-64bd841c-791c-4a7c-b0d1-1cd4419cf2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:32.191793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 19:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:32.230394 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f920d16-d5d9-4e2f-903b-9f4cffd90615 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:32.232041 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6f920d16-d5d9-4e2f-903b-9f4cffd90615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:32.336412 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f920d16-d5d9-4e2f-903b-9f4cffd90615 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:32.384665 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:32.409664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f3113-9813-4035-93e4-04eda4c25fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:32.410311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f3113-9813-4035-93e4-04eda4c25fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:32.410592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f3113-9813-4035-93e4-04eda4c25fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:32.410864 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f3113-9813-4035-93e4-04eda4c25fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:32.412516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5c6f3113-9813-4035-93e4-04eda4c25fe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:32.530365 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f3113-9813-4035-93e4-04eda4c25fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:32.531856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f3113-9813-4035-93e4-04eda4c25fe5 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:32.549871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 731/1455] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:34.615393 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790a27d-7cf1-4cfb-81f6-d8cbd464a420 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:34.615977 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790a27d-7cf1-4cfb-81f6-d8cbd464a420 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:34.615977 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790a27d-7cf1-4cfb-81f6-d8cbd464a420 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:34.617315 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 19:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:34.623237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13a55c-9da3-4b4e-ae58-e8282972ee75 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:34.623689 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13a55c-9da3-4b4e-ae58-e8282972ee75 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:34.623808 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13a55c-9da3-4b4e-ae58-e8282972ee75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:34.688317 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-cb13a55c-9da3-4b4e-ae58-e8282972ee75 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:34.689516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cb13a55c-9da3-4b4e-ae58-e8282972ee75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:34.727895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:34.800543 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 19:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:34.838582 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790a27d-7cf1-4cfb-81f6-d8cbd464a420 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:34.840171 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e790a27d-7cf1-4cfb-81f6-d8cbd464a420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:34.945508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790a27d-7cf1-4cfb-81f6-d8cbd464a420 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:34.974416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:35.023060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02329ed3-f9b9-4c7e-a278-ae82f7bc8691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:35.023747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02329ed3-f9b9-4c7e-a278-ae82f7bc8691 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:35.024038 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02329ed3-f9b9-4c7e-a278-ae82f7bc8691 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:35.024317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02329ed3-f9b9-4c7e-a278-ae82f7bc8691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:35.030263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-02329ed3-f9b9-4c7e-a278-ae82f7bc8691 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:35.148138 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02329ed3-f9b9-4c7e-a278-ae82f7bc8691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:35.149591 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02329ed3-f9b9-4c7e-a278-ae82f7bc8691 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:35.169331 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 726/1458] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:37.277487 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fc0e9-c8ba-47d8-970c-9b7e8e3deca3 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:37.278025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fc0e9-c8ba-47d8-970c-9b7e8e3deca3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:37.278025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fc0e9-c8ba-47d8-970c-9b7e8e3deca3 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:37.279408 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 19:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:37.284674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8468e0-1484-4b5b-9904-a7412cd35ffd None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:37.285105 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8468e0-1484-4b5b-9904-a7412cd35ffd None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:37.285234 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8468e0-1484-4b5b-9904-a7412cd35ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:37.349584 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4a8468e0-1484-4b5b-9904-a7412cd35ffd None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:37.350788 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4a8468e0-1484-4b5b-9904-a7412cd35ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:37.359642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:37.449454 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 19:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:37.487723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362fc0e9-c8ba-47d8-970c-9b7e8e3deca3 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:37.489238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-362fc0e9-c8ba-47d8-970c-9b7e8e3deca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:37.590487 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362fc0e9-c8ba-47d8-970c-9b7e8e3deca3 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:37.643439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:37.661954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad94ed-76a3-4edc-a6b8-6e8596e03daf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:37.662599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad94ed-76a3-4edc-a6b8-6e8596e03daf service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:37.662881 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad94ed-76a3-4edc-a6b8-6e8596e03daf service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:37.663185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad94ed-76a3-4edc-a6b8-6e8596e03daf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:37.664793 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9aad94ed-76a3-4edc-a6b8-6e8596e03daf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:37.770205 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad94ed-76a3-4edc-a6b8-6e8596e03daf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:37.771934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad94ed-76a3-4edc-a6b8-6e8596e03daf service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:37.791884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 734/1461] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:39.834101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f54866-c342-4921-bdf2-f85bc03682e1 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:39.834671 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f54866-c342-4921-bdf2-f85bc03682e1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:39.834771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f54866-c342-4921-bdf2-f85bc03682e1 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:39.836135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 19:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:39.841381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02a53b-c175-430e-8e9b-ba83b416f529 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:39.841808 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02a53b-c175-430e-8e9b-ba83b416f529 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:39.841945 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02a53b-c175-430e-8e9b-ba83b416f529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:39.906552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-df02a53b-c175-430e-8e9b-ba83b416f529 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:39.907790 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-df02a53b-c175-430e-8e9b-ba83b416f529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:39.945394 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:40.014622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 19:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:40.052012 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f54866-c342-4921-bdf2-f85bc03682e1 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:40.053565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-95f54866-c342-4921-bdf2-f85bc03682e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:40.157588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f54866-c342-4921-bdf2-f85bc03682e1 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:40.195377 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:40.233825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8341116-0c9b-4ac9-8337-c8f5aecf7565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:40.234472 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8341116-0c9b-4ac9-8337-c8f5aecf7565 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:40.234752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8341116-0c9b-4ac9-8337-c8f5aecf7565 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:40.235024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8341116-0c9b-4ac9-8337-c8f5aecf7565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:40.236690 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d8341116-0c9b-4ac9-8337-c8f5aecf7565 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:40.342778 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8341116-0c9b-4ac9-8337-c8f5aecf7565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:40.344244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8341116-0c9b-4ac9-8337-c8f5aecf7565 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:40.362740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 729/1464] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:42.376024 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ea4c6-12da-4861-879c-927b9ed6a765 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:42.376580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ea4c6-12da-4861-879c-927b9ed6a765 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:42.376580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ea4c6-12da-4861-879c-927b9ed6a765 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:42.378054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 19:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:42.383774 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19afbd-0b02-4e4a-82da-33e76fee47f7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:42.384178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19afbd-0b02-4e4a-82da-33e76fee47f7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:42.384335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19afbd-0b02-4e4a-82da-33e76fee47f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:42.451916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-5d19afbd-0b02-4e4a-82da-33e76fee47f7 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:42.453130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5d19afbd-0b02-4e4a-82da-33e76fee47f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:42.470921 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:42.551332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 19:01:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:42.587637 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ea4c6-12da-4861-879c-927b9ed6a765 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:42.590134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b10ea4c6-12da-4861-879c-927b9ed6a765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:42.691983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ea4c6-12da-4861-879c-927b9ed6a765 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:42.762899 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:42.769411 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa732c-bdbb-4fbb-a3c0-c8b86f8beb20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:42.770077 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa732c-bdbb-4fbb-a3c0-c8b86f8beb20 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:42.770359 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa732c-bdbb-4fbb-a3c0-c8b86f8beb20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:42.770636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa732c-bdbb-4fbb-a3c0-c8b86f8beb20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:42.772578 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-30aa732c-bdbb-4fbb-a3c0-c8b86f8beb20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:42.882574 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa732c-bdbb-4fbb-a3c0-c8b86f8beb20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:42.884046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa732c-bdbb-4fbb-a3c0-c8b86f8beb20 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:42.902444 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 737/1467] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:44.939952 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7334276-5d0e-4d1c-9019-28f8cba9615e None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:44.940427 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7334276-5d0e-4d1c-9019-28f8cba9615e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:44.940536 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7334276-5d0e-4d1c-9019-28f8cba9615e None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:44.941945 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 19:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:01:44.947189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94355de-bc1b-48f8-9218-11990583faca None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:44.947603 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94355de-bc1b-48f8-9218-11990583faca None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:44.947781 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94355de-bc1b-48f8-9218-11990583faca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:45.015264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c94355de-bc1b-48f8-9218-11990583faca None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:01:45.016477 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c94355de-bc1b-48f8-9218-11990583faca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:45.065555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:45.114152 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 19:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:01:45.150117 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7334276-5d0e-4d1c-9019-28f8cba9615e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:01:45.153940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d7334276-5d0e-4d1c-9019-28f8cba9615e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:45.255632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7334276-5d0e-4d1c-9019-28f8cba9615e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:01:45.304817 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:45.331212 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f82aa-754d-4b11-b201-c39a5eb44862 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:01:45.331697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f82aa-754d-4b11-b201-c39a5eb44862 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:01:45.331985 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f82aa-754d-4b11-b201-c39a5eb44862 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:01:45.332269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f82aa-754d-4b11-b201-c39a5eb44862 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:01:45.333912 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ba1f82aa-754d-4b11-b201-c39a5eb44862 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:01:45.445802 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f82aa-754d-4b11-b201-c39a5eb44862 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:01:45.447277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f82aa-754d-4b11-b201-c39a5eb44862 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:01:45.466511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 732/1470] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:01:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:01:55.112563 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=22 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:01:55.463388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=66 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:02:05.363757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac0b67-802c-4460-9eff-05832303d8a7 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:05.364369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac0b67-802c-4460-9eff-05832303d8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:05.364369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac0b67-802c-4460-9eff-05832303d8a7 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:05.365734 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 739/1471] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:02:05.371682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223072c5-1c5a-4202-97e8-b9716eda68a6 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:05.372268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223072c5-1c5a-4202-97e8-b9716eda68a6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:05.372268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223072c5-1c5a-4202-97e8-b9716eda68a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:05.449846 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-223072c5-1c5a-4202-97e8-b9716eda68a6 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:02:05.451083 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-223072c5-1c5a-4202-97e8-b9716eda68a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:05.563859 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 733/1472] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:02:05.615208 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406fe0e-f844-4888-a603-8ee4ac9eaeaf None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:05.615440 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406fe0e-f844-4888-a603-8ee4ac9eaeaf None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:05.615729 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406fe0e-f844-4888-a603-8ee4ac9eaeaf None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:05.617168 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 740/1473] 10.222.0.69 () {54 vars in 963 bytes} [Thu Jul 14 19:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:02:05.622217 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc44c8d-d5c8-4193-9a76-5eaf597faff7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:05.622513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc44c8d-d5c8-4193-9a76-5eaf597faff7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:05.622797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc44c8d-d5c8-4193-9a76-5eaf597faff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:05.723738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7bc44c8d-d5c8-4193-9a76-5eaf597faff7 None None] MFA Rules not processed for user `e5d76eae62b54521a95f36284118fada`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:02:05.724987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7bc44c8d-d5c8-4193-9a76-5eaf597faff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:05.749282 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:02:05.814160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 734/1474] 10.222.0.69 () {58 vars in 1060 bytes} [Thu Jul 14 19:02:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:02:05.821661 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406fe0e-f844-4888-a603-8ee4ac9eaeaf None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:02:05.823985 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b406fe0e-f844-4888-a603-8ee4ac9eaeaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:05.950797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406fe0e-f844-4888-a603-8ee4ac9eaeaf None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:02:06.033044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9ee09-3961-4917-bfcd-7aca026fe5b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5d76eae62b54521a95f36284118fada', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:02:06.033721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9ee09-3961-4917-bfcd-7aca026fe5b8 service glance] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:06.034010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9ee09-3961-4917-bfcd-7aca026fe5b8 service glance] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:06.034288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9ee09-3961-4917-bfcd-7aca026fe5b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:06.036123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-08e9ee09-3961-4917-bfcd-7aca026fe5b8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:06.094107 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:02:06.152497 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9ee09-3961-4917-bfcd-7aca026fe5b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:02:06.153956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9ee09-3961-4917-bfcd-7aca026fe5b8 service glance] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:02:06.173269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 741/1475] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:02:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:02:08.354835 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098e805-c53b-418f-89c9-5e6631d70124 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:08.355384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098e805-c53b-418f-89c9-5e6631d70124 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:08.355482 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098e805-c53b-418f-89c9-5e6631d70124 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:08.356849 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 735/1476] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:02:08.362208 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97490eb-22cb-4b33-a0c6-75b57706f1fc None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:08.362588 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97490eb-22cb-4b33-a0c6-75b57706f1fc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:08.362795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97490eb-22cb-4b33-a0c6-75b57706f1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:08.429213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d97490eb-22cb-4b33-a0c6-75b57706f1fc None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:02:08.430424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d97490eb-22cb-4b33-a0c6-75b57706f1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:08.532232 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 742/1477] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:02:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:02:08.558384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098e805-c53b-418f-89c9-5e6631d70124 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:02:08.559965 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a098e805-c53b-418f-89c9-5e6631d70124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:08.691279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098e805-c53b-418f-89c9-5e6631d70124 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:02:08.740639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:02:08.790138 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23178a6-cb20-49c8-8214-0bf4b9d367ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:02:08.790802 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23178a6-cb20-49c8-8214-0bf4b9d367ba admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:08.791631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23178a6-cb20-49c8-8214-0bf4b9d367ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:08.791900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23178a6-cb20-49c8-8214-0bf4b9d367ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:08.792655 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23178a6-cb20-49c8-8214-0bf4b9d367ba admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:02:08.794377 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: WARNING py.warnings [None req-e23178a6-cb20-49c8-8214-0bf4b9d367ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 19:02:08.794377 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: warnings.warn(msg) Jul 14 19:02:08.794377 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: Jul 14 19:02:08.795540 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23178a6-cb20-49c8-8214-0bf4b9d367ba admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:02:08.814762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 736/1478] 10.222.0.69 () {56 vars in 1108 bytes} [Thu Jul 14 19:02:08 2022] GET /identity/v3/projects => generated 2553 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:02:10.877658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6ed3d-b3d9-40b2-91a0-febff947785d None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:10.878217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6ed3d-b3d9-40b2-91a0-febff947785d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:10.878217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6ed3d-b3d9-40b2-91a0-febff947785d None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:10.879665 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 743/1479] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:02:10.885476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe3db2-7937-4414-8842-42fecacaeadf None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:10.885921 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe3db2-7937-4414-8842-42fecacaeadf None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:10.886036 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe3db2-7937-4414-8842-42fecacaeadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:10.951842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d5fe3db2-7937-4414-8842-42fecacaeadf None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:02:10.953058 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d5fe3db2-7937-4414-8842-42fecacaeadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:11.052156 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 737/1480] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:02:11.095376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6ed3d-b3d9-40b2-91a0-febff947785d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:02:11.096894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-62a6ed3d-b3d9-40b2-91a0-febff947785d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:11.106564 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:02:11.240232 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6ed3d-b3d9-40b2-91a0-febff947785d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:02:11.320735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:02:11.353861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74bc89-de69-4c40-9689-be34c1af70c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:02:11.354512 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74bc89-de69-4c40-9689-be34c1af70c8 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:11.354792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74bc89-de69-4c40-9689-be34c1af70c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:11.355066 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74bc89-de69-4c40-9689-be34c1af70c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:11.356792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ef74bc89-de69-4c40-9689-be34c1af70c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:11.444651 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74bc89-de69-4c40-9689-be34c1af70c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:02:11.446173 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74bc89-de69-4c40-9689-be34c1af70c8 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:02:11.464267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 744/1481] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:02:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:02:13.689155 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8834f81-8da3-4811-ba81-c0b1b68efd71 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:13.690012 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8834f81-8da3-4811-ba81-c0b1b68efd71 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:13.690321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8834f81-8da3-4811-ba81-c0b1b68efd71 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:13.691977 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 738/1482] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:02:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:02:13.698202 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae730f45-73df-4e69-b079-d06fed58a65a None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:13.698771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae730f45-73df-4e69-b079-d06fed58a65a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:13.698771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae730f45-73df-4e69-b079-d06fed58a65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:13.766335 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ae730f45-73df-4e69-b079-d06fed58a65a None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:02:13.767624 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ae730f45-73df-4e69-b079-d06fed58a65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:13.814924 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:02:13.868975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 745/1483] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:02:13.906588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8834f81-8da3-4811-ba81-c0b1b68efd71 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:02:13.908205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d8834f81-8da3-4811-ba81-c0b1b68efd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:13.931979 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:02:14.019871 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8834f81-8da3-4811-ba81-c0b1b68efd71 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:02:14.095524 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aeecfb-ac6c-4f2c-ad92-bd1d253e2e32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:02:14.096176 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aeecfb-ac6c-4f2c-ad92-bd1d253e2e32 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:14.096459 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aeecfb-ac6c-4f2c-ad92-bd1d253e2e32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:14.096735 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aeecfb-ac6c-4f2c-ad92-bd1d253e2e32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:14.098352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d8aeecfb-ac6c-4f2c-ad92-bd1d253e2e32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:14.172431 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:02:14.208144 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aeecfb-ac6c-4f2c-ad92-bd1d253e2e32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:02:14.209520 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aeecfb-ac6c-4f2c-ad92-bd1d253e2e32 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:02:14.228207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 739/1484] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:02:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:02:16.314230 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f7c2d-db3a-4d7f-85f2-f104189a214c None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:16.314794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f7c2d-db3a-4d7f-85f2-f104189a214c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:16.314794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f7c2d-db3a-4d7f-85f2-f104189a214c None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:16.316118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 746/1485] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:02:16.321379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5e370-606f-4ddb-86c4-f1c388be0837 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:16.321883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5e370-606f-4ddb-86c4-f1c388be0837 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:16.321993 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5e370-606f-4ddb-86c4-f1c388be0837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:16.387017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a4e5e370-606f-4ddb-86c4-f1c388be0837 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:02:16.388291 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a4e5e370-606f-4ddb-86c4-f1c388be0837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:16.491777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 740/1486] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:02:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:02:16.530134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f7c2d-db3a-4d7f-85f2-f104189a214c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:02:16.531716 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4c8f7c2d-db3a-4d7f-85f2-f104189a214c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:16.638149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f7c2d-db3a-4d7f-85f2-f104189a214c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:02:16.710295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:02:16.712078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf4dbf3-0c77-4f59-89d1-91f1fc67df88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:02:16.713347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4dbf3-0c77-4f59-89d1-91f1fc67df88 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:16.713635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4dbf3-0c77-4f59-89d1-91f1fc67df88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:16.713970 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4dbf3-0c77-4f59-89d1-91f1fc67df88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:16.715659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8bf4dbf3-0c77-4f59-89d1-91f1fc67df88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:16.838712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf4dbf3-0c77-4f59-89d1-91f1fc67df88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:02:16.840206 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf4dbf3-0c77-4f59-89d1-91f1fc67df88 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:02:16.858645 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 747/1487] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:02:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:02:18.990925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bf885-b667-48bc-8ecd-6f4067ed995b None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:18.991530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bf885-b667-48bc-8ecd-6f4067ed995b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:18.991530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bf885-b667-48bc-8ecd-6f4067ed995b None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:18.992878 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 741/1488] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:02:18.998519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cee992-2b02-4b0a-8d2f-7fb4fd0ec873 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:18.998998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cee992-2b02-4b0a-8d2f-7fb4fd0ec873 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:18.999114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cee992-2b02-4b0a-8d2f-7fb4fd0ec873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:19.069366 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-20cee992-2b02-4b0a-8d2f-7fb4fd0ec873 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:02:19.071501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-20cee992-2b02-4b0a-8d2f-7fb4fd0ec873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:19.123362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:02:19.167927 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 748/1489] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:02:19.213438 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bf885-b667-48bc-8ecd-6f4067ed995b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:02:19.214963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b10bf885-b667-48bc-8ecd-6f4067ed995b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:19.250346 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:02:19.319424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bf885-b667-48bc-8ecd-6f4067ed995b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:02:19.395548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d85c0-c320-45cd-b6af-894e652b5c35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:02:19.396456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d85c0-c320-45cd-b6af-894e652b5c35 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:19.396747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d85c0-c320-45cd-b6af-894e652b5c35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:19.397025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d85c0-c320-45cd-b6af-894e652b5c35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:19.398673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a40d85c0-c320-45cd-b6af-894e652b5c35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:19.479519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:02:19.508791 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d85c0-c320-45cd-b6af-894e652b5c35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:02:19.510799 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d85c0-c320-45cd-b6af-894e652b5c35 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:02:19.529865 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 742/1490] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:02:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:02:21.919563 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729058e4-8021-4968-b535-3a4bb0199806 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:21.920132 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729058e4-8021-4968-b535-3a4bb0199806 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:21.920132 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729058e4-8021-4968-b535-3a4bb0199806 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:21.921431 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 749/1491] 10.222.0.69 () {54 vars in 926 bytes} [Thu Jul 14 19:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:02:21.926667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14962c-7b78-44f9-8553-6f110dd14ab8 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:21.927125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14962c-7b78-44f9-8553-6f110dd14ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:21.927301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14962c-7b78-44f9-8553-6f110dd14ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:21.997035 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-6c14962c-7b78-44f9-8553-6f110dd14ab8 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:02:21.998296 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6c14962c-7b78-44f9-8553-6f110dd14ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:22.096534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 743/1492] 10.222.0.69 () {58 vars in 1023 bytes} [Thu Jul 14 19:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:02:22.136279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729058e4-8021-4968-b535-3a4bb0199806 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:02:22.140629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-729058e4-8021-4968-b535-3a4bb0199806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:22.246359 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729058e4-8021-4968-b535-3a4bb0199806 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:02:22.318846 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa06820-1d73-4373-aeac-b01e596d1ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:02:22.319526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa06820-1d73-4373-aeac-b01e596d1ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:02:22.319816 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa06820-1d73-4373-aeac-b01e596d1ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:02:22.320091 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa06820-1d73-4373-aeac-b01e596d1ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:02:22.322339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0aa06820-1d73-4373-aeac-b01e596d1ab2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:02:22.331530 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:02:22.434347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa06820-1d73-4373-aeac-b01e596d1ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:02:22.435809 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa06820-1d73-4373-aeac-b01e596d1ab2 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:02:22.454447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 750/1493] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:02:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:02:32.095013 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=82 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:02:32.452087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=50 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:03:59.070599 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c20cf-19ee-4a0d-b62a-b75dcc11ab0a None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:03:59.071227 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c20cf-19ee-4a0d-b62a-b75dcc11ab0a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:03:59.071227 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c20cf-19ee-4a0d-b62a-b75dcc11ab0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:03:59.147026 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4e9c20cf-19ee-4a0d-b62a-b75dcc11ab0a None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:03:59.148744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4e9c20cf-19ee-4a0d-b62a-b75dcc11ab0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:03:59.253907 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 744/1494] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:03:59.261954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7fd9c-29af-44d3-a744-5a57c26b5f7a None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:03:59.262487 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7fd9c-29af-44d3-a744-5a57c26b5f7a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:03:59.262593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7fd9c-29af-44d3-a744-5a57c26b5f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:03:59.338111 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-f9c7fd9c-29af-44d3-a744-5a57c26b5f7a None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:03:59.339423 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f9c7fd9c-29af-44d3-a744-5a57c26b5f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:03:59.437581 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 751/1495] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:03:59.442095 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c20cf-19ee-4a0d-b62a-b75dcc11ab0a None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:03:59.443324 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4e9c20cf-19ee-4a0d-b62a-b75dcc11ab0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:03:59.454601 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c20cf-19ee-4a0d-b62a-b75dcc11ab0a None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:03:59.517807 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:03:59.551551 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814ea77-2c2b-4b4e-8ab0-3ab1e98a1cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:03:59.552224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814ea77-2c2b-4b4e-8ab0-3ab1e98a1cec admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:03:59.552506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814ea77-2c2b-4b4e-8ab0-3ab1e98a1cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:03:59.552781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814ea77-2c2b-4b4e-8ab0-3ab1e98a1cec admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:03:59.553504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814ea77-2c2b-4b4e-8ab0-3ab1e98a1cec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:03:59.553810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814ea77-2c2b-4b4e-8ab0-3ab1e98a1cec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:03:59.554825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: WARNING py.warnings [None req-8814ea77-2c2b-4b4e-8ab0-3ab1e98a1cec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 19:03:59.554825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: warnings.warn(msg) Jul 14 19:03:59.554825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: Jul 14 19:03:59.555745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814ea77-2c2b-4b4e-8ab0-3ab1e98a1cec admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:03:59.567590 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 745/1496] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:03:59.576748 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7fd9c-29af-44d3-a744-5a57c26b5f7a None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:03:59.586498 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7fd9c-29af-44d3-a744-5a57c26b5f7a None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:03:59.652262 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:03:59.686577 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d7f2b-281e-4be0-8538-93ce2e426671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:03:59.687278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d7f2b-281e-4be0-8538-93ce2e426671 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:03:59.687564 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d7f2b-281e-4be0-8538-93ce2e426671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:03:59.687839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d7f2b-281e-4be0-8538-93ce2e426671 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:03:59.688522 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d7f2b-281e-4be0-8538-93ce2e426671 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:03:59.690271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: WARNING py.warnings [None req-bc9d7f2b-281e-4be0-8538-93ce2e426671 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 19:03:59.690271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: warnings.warn(msg) Jul 14 19:03:59.690271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: Jul 14 19:03:59.691576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d7f2b-281e-4be0-8538-93ce2e426671 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:03:59.807304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 752/1497] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:03:59 2022] POST /identity/v3/projects => generated 379 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:03:59.808419 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814ea77-2c2b-4b4e-8ab0-3ab1e98a1cec admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:03:59.809956 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8814ea77-2c2b-4b4e-8ab0-3ab1e98a1cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:03:59.844526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814ea77-2c2b-4b4e-8ab0-3ab1e98a1cec admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:03:59.939297 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894459bd-2119-4ba1-bb77-f62b558859fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:03:59.939965 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894459bd-2119-4ba1-bb77-f62b558859fd admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:03:59.940249 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894459bd-2119-4ba1-bb77-f62b558859fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:03:59.940525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894459bd-2119-4ba1-bb77-f62b558859fd admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:03:59.941205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894459bd-2119-4ba1-bb77-f62b558859fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:03:59.942216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: WARNING py.warnings [None req-894459bd-2119-4ba1-bb77-f62b558859fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 19:03:59.942216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: warnings.warn(msg) Jul 14 19:03:59.942216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: Jul 14 19:03:59.943435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894459bd-2119-4ba1-bb77-f62b558859fd admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:04:00.047636 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 746/1498] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:03:59 2022] POST /identity/v3/users => generated 290 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:04:00.051875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d7f2b-281e-4be0-8538-93ce2e426671 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:04:00.053391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bc9d7f2b-281e-4be0-8538-93ce2e426671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:04:00.117011 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d7f2b-281e-4be0-8538-93ce2e426671 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:04:00.203853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:04:00.214194 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c889a2-2062-4c39-848c-375832785ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:04:00.215929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c889a2-2062-4c39-848c-375832785ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:04:00.215929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c889a2-2062-4c39-848c-375832785ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:04:00.215929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c889a2-2062-4c39-848c-375832785ba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:04:00.216766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c889a2-2062-4c39-848c-375832785ba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:04:00.217214 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: WARNING py.warnings [None req-b4c889a2-2062-4c39-848c-375832785ba7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 19:04:00.217214 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: warnings.warn(msg) Jul 14 19:04:00.217214 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: Jul 14 19:04:00.218196 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c889a2-2062-4c39-848c-375832785ba7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:04:00.231057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 753/1499] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:04:00.235127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894459bd-2119-4ba1-bb77-f62b558859fd admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:04:00.236689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-894459bd-2119-4ba1-bb77-f62b558859fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:04:00.248767 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894459bd-2119-4ba1-bb77-f62b558859fd admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:04:00.274639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:04:00.351661 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d8ddd-e95e-425f-b3a3-6a31a092ae64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:04:00.352340 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d8ddd-e95e-425f-b3a3-6a31a092ae64 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:04:00.352623 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d8ddd-e95e-425f-b3a3-6a31a092ae64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:04:00.352899 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d8ddd-e95e-425f-b3a3-6a31a092ae64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:04:00.353635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d8ddd-e95e-425f-b3a3-6a31a092ae64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:04:00.354648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: WARNING py.warnings [None req-0b6d8ddd-e95e-425f-b3a3-6a31a092ae64 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 19:04:00.354648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: warnings.warn(msg) Jul 14 19:04:00.354648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: Jul 14 19:04:00.355590 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d8ddd-e95e-425f-b3a3-6a31a092ae64 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:04:00.369775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 747/1500] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:04:00.373966 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c889a2-2062-4c39-848c-375832785ba7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:04:00.375559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b4c889a2-2062-4c39-848c-375832785ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:04:00.409019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c889a2-2062-4c39-848c-375832785ba7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:04:00.505073 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766de7f-75c1-4048-882b-58dd01681168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:04:00.505970 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766de7f-75c1-4048-882b-58dd01681168 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:04:00.506258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766de7f-75c1-4048-882b-58dd01681168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:04:00.506590 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766de7f-75c1-4048-882b-58dd01681168 admin admin] PATH_INFO: `/v3/projects/2e6e44021d994157af57d2ebbddddfd5/users/173c3bdb349e4f57a42c997c3ab0fb69/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:04:00.529003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766de7f-75c1-4048-882b-58dd01681168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e6e44021d994157af57d2ebbddddfd5, user_id=173c3bdb349e4f57a42c997c3ab0fb69, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:04:00.530131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: WARNING py.warnings [None req-6766de7f-75c1-4048-882b-58dd01681168 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 19:04:00.530131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: warnings.warn(msg) Jul 14 19:04:00.530131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: Jul 14 19:04:00.532114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766de7f-75c1-4048-882b-58dd01681168 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:04:00.671689 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 754/1501] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:04:00 2022] PUT /identity/v3/projects/2e6e44021d994157af57d2ebbddddfd5/users/173c3bdb349e4f57a42c997c3ab0fb69/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:04:00.684047 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8b7ff-26f9-4ece-aaf6-1bc3f9557bcd None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:04:00.684340 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8b7ff-26f9-4ece-aaf6-1bc3f9557bcd None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:04:00.684621 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8b7ff-26f9-4ece-aaf6-1bc3f9557bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:04:00.745090 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ebb8b7ff-26f9-4ece-aaf6-1bc3f9557bcd None None] MFA Rules not processed for user `173c3bdb349e4f57a42c997c3ab0fb69`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:04:00.746311 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ebb8b7ff-26f9-4ece-aaf6-1bc3f9557bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:04:00.833304 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 748/1502] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:04:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:04:00.847246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888b012-f0f2-468e-8157-badd1299e047 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:04:00.847553 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888b012-f0f2-468e-8157-badd1299e047 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:04:00.847836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888b012-f0f2-468e-8157-badd1299e047 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:04:00.849253 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 755/1503] 10.222.0.69 () {54 vars in 963 bytes} [Thu Jul 14 19:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:04:00.854808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37875375-bd15-4ee5-97dd-be6881a0a8b6 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:04:00.855101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37875375-bd15-4ee5-97dd-be6881a0a8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:04:00.855417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37875375-bd15-4ee5-97dd-be6881a0a8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:04:00.885465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:04:00.932860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-37875375-bd15-4ee5-97dd-be6881a0a8b6 None None] MFA Rules not processed for user `31b09d72314349c6be0f906484895608`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:04:00.933995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-37875375-bd15-4ee5-97dd-be6881a0a8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:04:01.011350 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 749/1504] 10.222.0.69 () {58 vars in 1060 bytes} [Thu Jul 14 19:04:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:04:01.019183 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888b012-f0f2-468e-8157-badd1299e047 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:04:01.020757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e888b012-f0f2-468e-8157-badd1299e047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:04:01.073099 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:04:01.139076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888b012-f0f2-468e-8157-badd1299e047 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:04:01.215525 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62029426-7515-439e-83a8-6219bc02ce4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:04:01.216288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62029426-7515-439e-83a8-6219bc02ce4b service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:04:01.216582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62029426-7515-439e-83a8-6219bc02ce4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:04:01.216863 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62029426-7515-439e-83a8-6219bc02ce4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:04:01.218536 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-62029426-7515-439e-83a8-6219bc02ce4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:04:01.336668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62029426-7515-439e-83a8-6219bc02ce4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:04:01.338112 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62029426-7515-439e-83a8-6219bc02ce4b service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:04:01.342215 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:04:01.357939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 756/1505] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:04:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:04:01.402352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889853f3-e24f-4f35-876d-2ca111dca16a None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:04:01.402801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889853f3-e24f-4f35-876d-2ca111dca16a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:04:01.402899 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889853f3-e24f-4f35-876d-2ca111dca16a None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:04:01.404219 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 750/1506] 10.222.0.69 () {52 vars in 829 bytes} [Thu Jul 14 19:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:04:01.407827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62029426-7515-439e-83a8-6219bc02ce4b service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:04:01.409058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-62029426-7515-439e-83a8-6219bc02ce4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:04:01.513181 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62029426-7515-439e-83a8-6219bc02ce4b service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:04:01.609624 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2e852-f580-4f85-abba-14422e91b7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:04:01.610516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2e852-f580-4f85-abba-14422e91b7ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:04:01.610803 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2e852-f580-4f85-abba-14422e91b7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:04:01.611090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2e852-f580-4f85-abba-14422e91b7ec admin admin] PATH_INFO: `/v3/users/173c3bdb349e4f57a42c997c3ab0fb69` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:04:01.625426 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2e852-f580-4f85-abba-14422e91b7ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=173c3bdb349e4f57a42c997c3ab0fb69)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:04:01.626573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: WARNING py.warnings [None req-02d2e852-f580-4f85-abba-14422e91b7ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 19:04:01.626573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: warnings.warn(msg) Jul 14 19:04:01.626573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: Jul 14 19:04:01.627852 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2e852-f580-4f85-abba-14422e91b7ec admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:04:01.639154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:04:01.828051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-02d2e852-f580-4f85-abba-14422e91b7ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '173c3bdb349e4f57a42c997c3ab0fb69'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:04:01.829060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-02d2e852-f580-4f85-abba-14422e91b7ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '173c3bdb349e4f57a42c997c3ab0fb69'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:04:01.849932 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-02d2e852-f580-4f85-abba-14422e91b7ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '173c3bdb349e4f57a42c997c3ab0fb69'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:04:01.850445 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-02d2e852-f580-4f85-abba-14422e91b7ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '173c3bdb349e4f57a42c997c3ab0fb69'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:04:01.864152 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-02d2e852-f580-4f85-abba-14422e91b7ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '173c3bdb349e4f57a42c997c3ab0fb69'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:04:01.959198 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 757/1507] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:04:01 2022] DELETE /identity/v3/users/173c3bdb349e4f57a42c997c3ab0fb69 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:04:01.959437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760ad65-102a-4032-aeb5-e0b372f4ba19 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:04:01.959437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760ad65-102a-4032-aeb5-e0b372f4ba19 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:04:01.959437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760ad65-102a-4032-aeb5-e0b372f4ba19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:04:02.024062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d760ad65-102a-4032-aeb5-e0b372f4ba19 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:04:02.025275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d760ad65-102a-4032-aeb5-e0b372f4ba19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:04:02.102521 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:04:02.114983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 751/1508] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:04:02.125382 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2e852-f580-4f85-abba-14422e91b7ec admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:04:02.126612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-02d2e852-f580-4f85-abba-14422e91b7ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:04:02.231214 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2e852-f580-4f85-abba-14422e91b7ec admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:04:02.286861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:04:02.308609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374745e-e88f-414c-a83e-da95c3f776f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:04:02.309285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374745e-e88f-414c-a83e-da95c3f776f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:04:02.309582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374745e-e88f-414c-a83e-da95c3f776f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:04:02.309862 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374745e-e88f-414c-a83e-da95c3f776f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:04:02.311723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d374745e-e88f-414c-a83e-da95c3f776f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:04:02.432539 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374745e-e88f-414c-a83e-da95c3f776f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:04:02.433908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374745e-e88f-414c-a83e-da95c3f776f2 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:04:02.452427 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 758/1509] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:04:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:04:02.937238 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760ad65-102a-4032-aeb5-e0b372f4ba19 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:04:02.938468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d760ad65-102a-4032-aeb5-e0b372f4ba19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:04:02.951358 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760ad65-102a-4032-aeb5-e0b372f4ba19 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:04:03.055887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b22be-b6ed-4ee4-a581-ec9f12041a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:04:03.056760 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b22be-b6ed-4ee4-a581-ec9f12041a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:04:03.057047 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b22be-b6ed-4ee4-a581-ec9f12041a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:04:03.057327 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b22be-b6ed-4ee4-a581-ec9f12041a50 admin admin] PATH_INFO: `/v3/projects/2e6e44021d994157af57d2ebbddddfd5` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:04:03.068866 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b22be-b6ed-4ee4-a581-ec9f12041a50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e6e44021d994157af57d2ebbddddfd5)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:04:03.070477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: WARNING py.warnings [None req-8b0b22be-b6ed-4ee4-a581-ec9f12041a50 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 19:04:03.070477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: warnings.warn(msg) Jul 14 19:04:03.070477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: Jul 14 19:04:03.071716 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b22be-b6ed-4ee4-a581-ec9f12041a50 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:04:03.189581 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8b0b22be-b6ed-4ee4-a581-ec9f12041a50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e6e44021d994157af57d2ebbddddfd5', 'request_id': 'req-8b0b22be-b6ed-4ee4-a581-ec9f12041a50'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:04:03.270422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8b0b22be-b6ed-4ee4-a581-ec9f12041a50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e6e44021d994157af57d2ebbddddfd5', 'request_id': 'req-8b0b22be-b6ed-4ee4-a581-ec9f12041a50'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:04:03.278471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8b0b22be-b6ed-4ee4-a581-ec9f12041a50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e6e44021d994157af57d2ebbddddfd5', 'request_id': 'req-8b0b22be-b6ed-4ee4-a581-ec9f12041a50'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:04:03.311513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8b0b22be-b6ed-4ee4-a581-ec9f12041a50 admin admin] The token cache is being invalidate because project 2e6e44021d994157af57d2ebbddddfd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:04:03.311806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8b0b22be-b6ed-4ee4-a581-ec9f12041a50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:04:03.312239 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8b0b22be-b6ed-4ee4-a581-ec9f12041a50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:04:03.313719 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 752/1510] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:04:02 2022] DELETE /identity/v3/projects/2e6e44021d994157af57d2ebbddddfd5 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:04:12.450017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=62 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:04:13.309232 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:05:46.030081 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d890d-f540-4111-9c23-7181f1eecef6 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:05:46.030648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d890d-f540-4111-9c23-7181f1eecef6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:05:46.030648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d890d-f540-4111-9c23-7181f1eecef6 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:05:46.032081 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 19:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:05:46.037018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d55fe-a4fe-4877-b827-2d5d2475932b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:05:46.037455 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d55fe-a4fe-4877-b827-2d5d2475932b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:05:46.037582 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d55fe-a4fe-4877-b827-2d5d2475932b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:05:46.117865 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-744d55fe-a4fe-4877-b827-2d5d2475932b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:05:46.119717 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-744d55fe-a4fe-4877-b827-2d5d2475932b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:05:46.224786 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 19:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:05:46.377162 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d890d-f540-4111-9c23-7181f1eecef6 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:05:46.380175 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d46d890d-f540-4111-9c23-7181f1eecef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:05:46.498906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d890d-f540-4111-9c23-7181f1eecef6 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:05:46.504504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954eca00-0803-4371-aa73-dad463024bee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:05:46.505163 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954eca00-0803-4371-aa73-dad463024bee service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:05:46.505450 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954eca00-0803-4371-aa73-dad463024bee service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:05:46.505730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954eca00-0803-4371-aa73-dad463024bee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:05:46.515401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954eca00-0803-4371-aa73-dad463024bee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:05:46.517060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954eca00-0803-4371-aa73-dad463024bee service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:05:46.526873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 760/1513] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:05:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:05:47.368460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67697d4a-bc8b-4a67-83ea-d555c4f5c333 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:05:47.369046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67697d4a-bc8b-4a67-83ea-d555c4f5c333 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:05:47.369046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67697d4a-bc8b-4a67-83ea-d555c4f5c333 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:05:47.370594 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 19:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:05:47.375790 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d0284-1d22-44da-bbe7-89d3624767d9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:05:47.376249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d0284-1d22-44da-bbe7-89d3624767d9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:05:47.376350 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d0284-1d22-44da-bbe7-89d3624767d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:05:47.394745 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d17d0284-1d22-44da-bbe7-89d3624767d9 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:05:47.396016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d17d0284-1d22-44da-bbe7-89d3624767d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:05:47.404827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 19:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:05:47.554378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67697d4a-bc8b-4a67-83ea-d555c4f5c333 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:05:47.569265 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67697d4a-bc8b-4a67-83ea-d555c4f5c333 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:05:47.575214 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f3019c-c239-497c-8a10-ff6bba5d2993 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:05:47.575900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3019c-c239-497c-8a10-ff6bba5d2993 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:05:47.576187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3019c-c239-497c-8a10-ff6bba5d2993 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:05:47.576465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3019c-c239-497c-8a10-ff6bba5d2993 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:05:47.584752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3019c-c239-497c-8a10-ff6bba5d2993 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:05:47.586347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3019c-c239-497c-8a10-ff6bba5d2993 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:05:47.596595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 755/1516] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:05:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:05:47.909220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cafb54-df75-4d05-90b6-9f45995b6ef4 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:05:47.909570 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cafb54-df75-4d05-90b6-9f45995b6ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:05:47.919372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cafb54-df75-4d05-90b6-9f45995b6ef4 None None] PATH_INFO: `/v3` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:05:47.919372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 762/1517] 10.222.0.30 () {54 vars in 925 bytes} [Thu Jul 14 19:05:47 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:05:47.919777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea1478-d615-429a-a5d5-1a5b919bf0b3 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:05:47.919777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea1478-d615-429a-a5d5-1a5b919bf0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:05:47.919777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea1478-d615-429a-a5d5-1a5b919bf0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:05:47.978017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c5ea1478-d615-429a-a5d5-1a5b919bf0b3 None None] MFA Rules not processed for user `c577fa8670e34449a80de0c8e0a15a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:05:47.979250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c5ea1478-d615-429a-a5d5-1a5b919bf0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:05:47.996779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:05:48.054419 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 756/1518] 10.222.0.30 () {58 vars in 1015 bytes} [Thu Jul 14 19:05:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:05:57.389938 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=56 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:05:57.472726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e98f49-0f31-4f49-a496-95013008ee6c None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:05:57.473045 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e98f49-0f31-4f49-a496-95013008ee6c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:05:57.473335 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e98f49-0f31-4f49-a496-95013008ee6c None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:05:57.474929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 763/1519] 10.222.0.69 () {54 vars in 960 bytes} [Thu Jul 14 19:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:05:57.479750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5195ba-7272-4819-bb25-400850c53852 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:05:57.480232 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5195ba-7272-4819-bb25-400850c53852 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:05:57.480347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5195ba-7272-4819-bb25-400850c53852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:05:57.498822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7d5195ba-7272-4819-bb25-400850c53852 None None] MFA Rules not processed for user `c577fa8670e34449a80de0c8e0a15a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:05:57.500152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7d5195ba-7272-4819-bb25-400850c53852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:05:57.509054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 757/1520] 10.222.0.69 () {58 vars in 1057 bytes} [Thu Jul 14 19:05:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:05:57.517169 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e98f49-0f31-4f49-a496-95013008ee6c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:05:57.530617 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e98f49-0f31-4f49-a496-95013008ee6c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:05:57.535726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34481ff-dc64-4fce-b958-cca756c73619 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c577fa8670e34449a80de0c8e0a15a8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:05:57.536363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34481ff-dc64-4fce-b958-cca756c73619 service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:05:57.536655 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34481ff-dc64-4fce-b958-cca756c73619 service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:05:57.536936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34481ff-dc64-4fce-b958-cca756c73619 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:05:57.545536 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34481ff-dc64-4fce-b958-cca756c73619 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:05:57.547134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34481ff-dc64-4fce-b958-cca756c73619 service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:05:57.556830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 764/1521] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:05:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:05:58.836660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1aec67-6da1-4732-8956-5b27253473ea None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:05:58.837233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1aec67-6da1-4732-8956-5b27253473ea None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:05:58.837233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1aec67-6da1-4732-8956-5b27253473ea None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:05:58.838604 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 19:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:05:58.844262 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcf239-2d79-45d1-9d82-d1d96ea6495f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:05:58.844680 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcf239-2d79-45d1-9d82-d1d96ea6495f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:05:58.844840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcf239-2d79-45d1-9d82-d1d96ea6495f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:05:58.862959 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-bffcf239-2d79-45d1-9d82-d1d96ea6495f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:05:58.864253 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bffcf239-2d79-45d1-9d82-d1d96ea6495f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:05:58.872078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 19:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:05:59.024325 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1aec67-6da1-4732-8956-5b27253473ea None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:05:59.040198 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1aec67-6da1-4732-8956-5b27253473ea None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:05:59.044567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1e022-b745-4ce7-b150-5616cf8061e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:05:59.045211 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1e022-b745-4ce7-b150-5616cf8061e2 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:05:59.045499 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1e022-b745-4ce7-b150-5616cf8061e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:05:59.045781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1e022-b745-4ce7-b150-5616cf8061e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:05:59.054255 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1e022-b745-4ce7-b150-5616cf8061e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:05:59.055883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1e022-b745-4ce7-b150-5616cf8061e2 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:05:59.066024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 759/1524] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:05:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:05:59.905620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724d5bf-629a-47dc-938a-5b654fb34b24 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:05:59.906247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724d5bf-629a-47dc-938a-5b654fb34b24 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:05:59.906247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724d5bf-629a-47dc-938a-5b654fb34b24 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:05:59.907608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 19:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:05:59.912371 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a589ac-096c-42c0-aa10-9702483cdd4b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:05:59.912778 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a589ac-096c-42c0-aa10-9702483cdd4b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:05:59.912935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a589ac-096c-42c0-aa10-9702483cdd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:05:59.931543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-86a589ac-096c-42c0-aa10-9702483cdd4b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:05:59.932747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-86a589ac-096c-42c0-aa10-9702483cdd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:05:59.940702 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 19:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:06:00.092463 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724d5bf-629a-47dc-938a-5b654fb34b24 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:06:00.106645 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724d5bf-629a-47dc-938a-5b654fb34b24 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:06:00.112092 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5719c-e057-426e-94d7-41db380a0b38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:06:00.112734 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5719c-e057-426e-94d7-41db380a0b38 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:06:00.113025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5719c-e057-426e-94d7-41db380a0b38 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:06:00.113337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5719c-e057-426e-94d7-41db380a0b38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:06:00.121887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5719c-e057-426e-94d7-41db380a0b38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:06:00.123529 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5719c-e057-426e-94d7-41db380a0b38 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:06:00.132988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 767/1527] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:06:04.098435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d00175-27de-4d04-92e1-b6187f1371f3 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:06:04.098983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d00175-27de-4d04-92e1-b6187f1371f3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:06:04.098983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d00175-27de-4d04-92e1-b6187f1371f3 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:06:04.100397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 19:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:06:04.105781 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca9792-9351-4951-a365-a4817b32abfe None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:06:04.106187 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca9792-9351-4951-a365-a4817b32abfe None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:06:04.106356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca9792-9351-4951-a365-a4817b32abfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:06:04.124209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-f8ca9792-9351-4951-a365-a4817b32abfe None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:06:04.125414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f8ca9792-9351-4951-a365-a4817b32abfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:06:04.133320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 19:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:06:04.175589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d00175-27de-4d04-92e1-b6187f1371f3 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:06:04.177324 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-68d00175-27de-4d04-92e1-b6187f1371f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:06:04.257537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d00175-27de-4d04-92e1-b6187f1371f3 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:06:04.261013 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cf0bb-553c-4d9e-9308-b8fdb9229ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:06:04.261660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cf0bb-553c-4d9e-9308-b8fdb9229ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:06:04.261944 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cf0bb-553c-4d9e-9308-b8fdb9229ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:06:04.262219 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cf0bb-553c-4d9e-9308-b8fdb9229ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:06:04.272419 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5cf0bb-553c-4d9e-9308-b8fdb9229ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:06:04.273849 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5cf0bb-553c-4d9e-9308-b8fdb9229ef5 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:06:04.283235 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 762/1530] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:06:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:06:06.364073 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492c949-a8b3-4f5c-8473-6e4b958a0ff9 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:06:06.364610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492c949-a8b3-4f5c-8473-6e4b958a0ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:06:06.364610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492c949-a8b3-4f5c-8473-6e4b958a0ff9 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:06:06.366009 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 19:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:06:06.371319 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd56652-e178-4c2a-b6d7-9e1074e5681f None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:06:06.371728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd56652-e178-4c2a-b6d7-9e1074e5681f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:06:06.371888 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd56652-e178-4c2a-b6d7-9e1074e5681f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:06:06.390699 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7fd56652-e178-4c2a-b6d7-9e1074e5681f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:06:06.391918 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7fd56652-e178-4c2a-b6d7-9e1074e5681f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:06:06.399943 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 19:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:06:06.436177 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492c949-a8b3-4f5c-8473-6e4b958a0ff9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:06:06.447055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492c949-a8b3-4f5c-8473-6e4b958a0ff9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:06:06.450606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121412d-9c3e-4720-9ebf-c135c7c8aee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:06:06.451922 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121412d-9c3e-4720-9ebf-c135c7c8aee5 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:06:06.452215 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121412d-9c3e-4720-9ebf-c135c7c8aee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:06:06.452495 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121412d-9c3e-4720-9ebf-c135c7c8aee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:06:06.462843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121412d-9c3e-4720-9ebf-c135c7c8aee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:06:06.464362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121412d-9c3e-4720-9ebf-c135c7c8aee5 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:06:06.474985 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 770/1533] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:06:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:06:10.276961 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719c673-aff2-4153-9eb8-42ba961094f7 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:06:10.277520 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719c673-aff2-4153-9eb8-42ba961094f7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:06:10.277520 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719c673-aff2-4153-9eb8-42ba961094f7 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:06:10.284680 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 19:06:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:06:10.284948 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653096a6-36c8-4ad2-8b2f-a3132588be18 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:06:10.285365 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653096a6-36c8-4ad2-8b2f-a3132588be18 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:06:10.285513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653096a6-36c8-4ad2-8b2f-a3132588be18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:06:10.305606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-653096a6-36c8-4ad2-8b2f-a3132588be18 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:06:10.306810 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-653096a6-36c8-4ad2-8b2f-a3132588be18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:06:10.314655 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 19:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:06:10.623064 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719c673-aff2-4153-9eb8-42ba961094f7 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:06:10.624852 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f719c673-aff2-4153-9eb8-42ba961094f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:06:10.692608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:06:10.706062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719c673-aff2-4153-9eb8-42ba961094f7 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:06:10.709633 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3cff2-24be-4614-975c-226593bfde41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:06:10.725917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3cff2-24be-4614-975c-226593bfde41 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:06:10.725917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3cff2-24be-4614-975c-226593bfde41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:06:10.725917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3cff2-24be-4614-975c-226593bfde41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:06:10.725917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3cff2-24be-4614-975c-226593bfde41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:06:10.725917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3cff2-24be-4614-975c-226593bfde41 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:06:10.731304 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 765/1536] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:06:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:06:12.787931 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13009d91-00cc-4a30-ab59-ab922d11255f None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:06:12.788475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13009d91-00cc-4a30-ab59-ab922d11255f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:06:12.788475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13009d91-00cc-4a30-ab59-ab922d11255f None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:06:12.789868 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 19:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:06:12.795960 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f23691-2e65-4d63-a0a4-f31bbea080fe None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:06:12.796390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f23691-2e65-4d63-a0a4-f31bbea080fe None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:06:12.796531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f23691-2e65-4d63-a0a4-f31bbea080fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:06:12.815665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f5f23691-2e65-4d63-a0a4-f31bbea080fe None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:06:12.816887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f5f23691-2e65-4d63-a0a4-f31bbea080fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:06:12.825647 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 19:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:06:13.138069 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13009d91-00cc-4a30-ab59-ab922d11255f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:06:13.139770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-13009d91-00cc-4a30-ab59-ab922d11255f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:06:13.152640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13009d91-00cc-4a30-ab59-ab922d11255f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:06:13.156624 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d919a73-4211-4a1c-ab2f-bfb1df5c0b18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:06:13.157258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d919a73-4211-4a1c-ab2f-bfb1df5c0b18 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:06:13.157546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d919a73-4211-4a1c-ab2f-bfb1df5c0b18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:06:13.157818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d919a73-4211-4a1c-ab2f-bfb1df5c0b18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:06:13.166486 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d919a73-4211-4a1c-ab2f-bfb1df5c0b18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:06:13.167939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d919a73-4211-4a1c-ab2f-bfb1df5c0b18 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:06:13.177781 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 773/1539] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:06:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:06:22.809851 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=11 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:06:23.173688 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=33 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:44.320012 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec4a63-b312-4a8e-bd20-f8422216f6f8 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:44.321308 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec4a63-b312-4a8e-bd20-f8422216f6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:44.321308 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec4a63-b312-4a8e-bd20-f8422216f6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:44.344803 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-58ec4a63-b312-4a8e-bd20-f8422216f6f8 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:44.346036 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-58ec4a63-b312-4a8e-bd20-f8422216f6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:44.360248 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 767/1540] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:44.369330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700133de-1f17-4ad7-ac1c-30dbd3a4e00e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:44.370505 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700133de-1f17-4ad7-ac1c-30dbd3a4e00e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:44.371022 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700133de-1f17-4ad7-ac1c-30dbd3a4e00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:44.398723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-700133de-1f17-4ad7-ac1c-30dbd3a4e00e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:44.400369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-700133de-1f17-4ad7-ac1c-30dbd3a4e00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:44.417342 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 774/1541] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:44.422366 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec4a63-b312-4a8e-bd20-f8422216f6f8 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:44.438893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec4a63-b312-4a8e-bd20-f8422216f6f8 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:44.447730 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce416a5c-06d4-46e5-b945-7fdad048a0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:44.448430 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce416a5c-06d4-46e5-b945-7fdad048a0dd admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:44.448765 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce416a5c-06d4-46e5-b945-7fdad048a0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:44.449053 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce416a5c-06d4-46e5-b945-7fdad048a0dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:44.449954 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce416a5c-06d4-46e5-b945-7fdad048a0dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:07:44.450271 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce416a5c-06d4-46e5-b945-7fdad048a0dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:44.456273 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce416a5c-06d4-46e5-b945-7fdad048a0dd admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:44.471526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 768/1542] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:07:44.480277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700133de-1f17-4ad7-ac1c-30dbd3a4e00e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:44.492252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700133de-1f17-4ad7-ac1c-30dbd3a4e00e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:44.502085 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594860dd-3c45-4874-9075-2dc01e50d807 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:44.505842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594860dd-3c45-4874-9075-2dc01e50d807 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:44.505842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594860dd-3c45-4874-9075-2dc01e50d807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:44.506063 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd127bbc-10df-4643-9f6c-cead9eebd8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:44.506063 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd127bbc-10df-4643-9f6c-cead9eebd8cb admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:44.506063 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd127bbc-10df-4643-9f6c-cead9eebd8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:44.506063 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd127bbc-10df-4643-9f6c-cead9eebd8cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:44.506874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd127bbc-10df-4643-9f6c-cead9eebd8cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:44.509529 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd127bbc-10df-4643-9f6c-cead9eebd8cb admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:44.536421 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-594860dd-3c45-4874-9075-2dc01e50d807 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:44.538107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-594860dd-3c45-4874-9075-2dc01e50d807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:44.560320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 769/1543] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:44.571425 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf789cd7-47fc-4e7c-ba71-beb146d48257 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:44.571662 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf789cd7-47fc-4e7c-ba71-beb146d48257 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:44.571897 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf789cd7-47fc-4e7c-ba71-beb146d48257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:44.613769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-cf789cd7-47fc-4e7c-ba71-beb146d48257 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:44.615665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-cf789cd7-47fc-4e7c-ba71-beb146d48257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:44.630930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 770/1544] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:07:44.631757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 775/1545] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:07:44 2022] POST /identity/v3/projects => generated 379 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:44.636595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf789cd7-47fc-4e7c-ba71-beb146d48257 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:44.639547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd127bbc-10df-4643-9f6c-cead9eebd8cb admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:44.653547 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf789cd7-47fc-4e7c-ba71-beb146d48257 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:44.758789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd127bbc-10df-4643-9f6c-cead9eebd8cb admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:44.772298 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91c0bd-2642-4177-90c9-bf2171f13abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:44.773405 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91c0bd-2642-4177-90c9-bf2171f13abe admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:44.773944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91c0bd-2642-4177-90c9-bf2171f13abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:44.774590 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91c0bd-2642-4177-90c9-bf2171f13abe admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:44.775537 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91c0bd-2642-4177-90c9-bf2171f13abe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:07:44.776123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91c0bd-2642-4177-90c9-bf2171f13abe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:44.777404 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: WARNING py.warnings [None req-7e91c0bd-2642-4177-90c9-bf2171f13abe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 19:07:44.777404 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: warnings.warn(msg) Jul 14 19:07:44.777404 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: Jul 14 19:07:44.779077 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91c0bd-2642-4177-90c9-bf2171f13abe admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:44.801120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 776/1546] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 19:07:44.804198 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964b0af-a1be-4771-a827-3d5c4e39d6db None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:44.804711 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964b0af-a1be-4771-a827-3d5c4e39d6db None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:44.805889 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964b0af-a1be-4771-a827-3d5c4e39d6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:44.844020 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7837c2b-0674-47cd-9fc8-a79725dcdfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:44.845092 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7837c2b-0674-47cd-9fc8-a79725dcdfd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:44.845275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7837c2b-0674-47cd-9fc8-a79725dcdfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:44.845559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7837c2b-0674-47cd-9fc8-a79725dcdfd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:44.846294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7837c2b-0674-47cd-9fc8-a79725dcdfd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:44.848139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6964b0af-a1be-4771-a827-3d5c4e39d6db None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:44.848486 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7837c2b-0674-47cd-9fc8-a79725dcdfd4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:44.849574 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6964b0af-a1be-4771-a827-3d5c4e39d6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:44.873860 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 777/1547] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:07:44.874892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964b0af-a1be-4771-a827-3d5c4e39d6db None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:44.897524 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964b0af-a1be-4771-a827-3d5c4e39d6db None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:44.907207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f09e8-d176-40af-9dcb-f6ccba3aa6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:44.907909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f09e8-d176-40af-9dcb-f6ccba3aa6ce admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:44.908211 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f09e8-d176-40af-9dcb-f6ccba3aa6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:44.908645 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f09e8-d176-40af-9dcb-f6ccba3aa6ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:44.909473 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f09e8-d176-40af-9dcb-f6ccba3aa6ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:44.911616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f09e8-d176-40af-9dcb-f6ccba3aa6ce admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:44.943839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 771/1548] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:07:44 2022] POST /identity/v3/users => generated 290 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:44.948168 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e50eba-897e-4c21-bcec-e80a484f0de7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:44.950213 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e50eba-897e-4c21-bcec-e80a484f0de7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:44.950509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e50eba-897e-4c21-bcec-e80a484f0de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:44.991404 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-67e50eba-897e-4c21-bcec-e80a484f0de7 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:44.994156 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-67e50eba-897e-4c21-bcec-e80a484f0de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:45.021049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 772/1549] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:45.022739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ec1fd-85c0-4043-97c5-b9f17617fe7e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.023034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ec1fd-85c0-4043-97c5-b9f17617fe7e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.023354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ec1fd-85c0-4043-97c5-b9f17617fe7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.042496 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 778/1550] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:07:44 2022] POST /identity/v3/projects => generated 391 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:45.043465 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f09e8-d176-40af-9dcb-f6ccba3aa6ce admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:45.053537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-de4ec1fd-85c0-4043-97c5-b9f17617fe7e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:45.055875 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-de4ec1fd-85c0-4043-97c5-b9f17617fe7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:45.121434 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f09e8-d176-40af-9dcb-f6ccba3aa6ce admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:45.126920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bc928-39bd-416d-85e4-fe35ba763b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:45.127667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bc928-39bd-416d-85e4-fe35ba763b5b admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.127981 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bc928-39bd-416d-85e4-fe35ba763b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.128260 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bc928-39bd-416d-85e4-fe35ba763b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.128996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28bc928-39bd-416d-85e4-fe35ba763b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:45.130615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28bc928-39bd-416d-85e4-fe35ba763b5b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:45.136912 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 773/1551] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:45.138458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5933d1-418e-4e6f-be2b-c5479deeea15 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.138777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5933d1-418e-4e6f-be2b-c5479deeea15 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.139057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5933d1-418e-4e6f-be2b-c5479deeea15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.141408 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 779/1552] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:45.142612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ad979-a418-46f8-ba18-90089a3fdd22 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.142932 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ad979-a418-46f8-ba18-90089a3fdd22 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.143218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ad979-a418-46f8-ba18-90089a3fdd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.158163 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-9d5933d1-418e-4e6f-be2b-c5479deeea15 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:45.159428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9d5933d1-418e-4e6f-be2b-c5479deeea15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:45.162505 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2f3ad979-a418-46f8-ba18-90089a3fdd22 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:45.163712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2f3ad979-a418-46f8-ba18-90089a3fdd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:45.167572 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 774/1553] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:45.169044 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cfde7-03b4-4c68-a384-fe145885b574 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.169338 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cfde7-03b4-4c68-a384-fe145885b574 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.169619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cfde7-03b4-4c68-a384-fe145885b574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.176682 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 780/1554] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:45.176682 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ad979-a418-46f8-ba18-90089a3fdd22 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:45.187313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ad979-a418-46f8-ba18-90089a3fdd22 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:45.192114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475241b-698a-444d-8e8e-5729230b1022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:45.192788 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475241b-698a-444d-8e8e-5729230b1022 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.192922 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-131cfde7-03b4-4c68-a384-fe145885b574 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:45.193073 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475241b-698a-444d-8e8e-5729230b1022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.193471 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475241b-698a-444d-8e8e-5729230b1022 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.194065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-131cfde7-03b4-4c68-a384-fe145885b574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:45.194174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475241b-698a-444d-8e8e-5729230b1022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:45.195210 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: WARNING py.warnings [None req-d475241b-698a-444d-8e8e-5729230b1022 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 19:07:45.195210 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: warnings.warn(msg) Jul 14 19:07:45.195210 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: Jul 14 19:07:45.196444 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475241b-698a-444d-8e8e-5729230b1022 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:45.202291 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 775/1555] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:45.203660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175dc9a-6523-40d1-bea3-c015fa8c37cf None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.203950 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175dc9a-6523-40d1-bea3-c015fa8c37cf None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.204231 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175dc9a-6523-40d1-bea3-c015fa8c37cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.227041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f175dc9a-6523-40d1-bea3-c015fa8c37cf None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:45.228318 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f175dc9a-6523-40d1-bea3-c015fa8c37cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:45.236813 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 776/1556] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:45.238266 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd67180-5521-44e7-9d2d-28bcaae9725e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.238412 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd67180-5521-44e7-9d2d-28bcaae9725e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.238689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd67180-5521-44e7-9d2d-28bcaae9725e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.257646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-8bd67180-5521-44e7-9d2d-28bcaae9725e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:45.258822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8bd67180-5521-44e7-9d2d-28bcaae9725e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:45.267470 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 777/1557] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:45.268858 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd67180-5521-44e7-9d2d-28bcaae9725e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:45.278684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd67180-5521-44e7-9d2d-28bcaae9725e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:45.283688 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6faa9-2266-43fc-a3ff-379d026b57df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:45.284358 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6faa9-2266-43fc-a3ff-379d026b57df admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.284595 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:45.284726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6faa9-2266-43fc-a3ff-379d026b57df admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.284920 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6faa9-2266-43fc-a3ff-379d026b57df admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.285656 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6faa9-2266-43fc-a3ff-379d026b57df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:07:45.285965 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6faa9-2266-43fc-a3ff-379d026b57df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:45.287701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6faa9-2266-43fc-a3ff-379d026b57df admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:45.292847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 781/1558] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:07:45 2022] POST /identity/v3/users => generated 296 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:45.292847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475241b-698a-444d-8e8e-5729230b1022 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:45.300807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475241b-698a-444d-8e8e-5729230b1022 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:45.301863 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 778/1559] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:07:45.302713 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6faa9-2266-43fc-a3ff-379d026b57df admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:45.311449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7b6e9-d04d-49da-9f49-c74447586a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:45.312283 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7b6e9-d04d-49da-9f49-c74447586a2a admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.312578 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7b6e9-d04d-49da-9f49-c74447586a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.312862 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7b6e9-d04d-49da-9f49-c74447586a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.313598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff7b6e9-d04d-49da-9f49-c74447586a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:45.315431 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff7b6e9-d04d-49da-9f49-c74447586a2a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:45.320553 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6faa9-2266-43fc-a3ff-379d026b57df admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:45.329945 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb9529-480a-40a0-88cb-1b6fb0f29d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:45.329945 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb9529-480a-40a0-88cb-1b6fb0f29d02 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.329945 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb9529-480a-40a0-88cb-1b6fb0f29d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.329945 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb9529-480a-40a0-88cb-1b6fb0f29d02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.330390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb9529-480a-40a0-88cb-1b6fb0f29d02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:07:45.330490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 782/1560] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:45.330594 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb9529-480a-40a0-88cb-1b6fb0f29d02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:45.330692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e1a76-59ac-4032-a158-652de061d929 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.330692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e1a76-59ac-4032-a158-652de061d929 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.330869 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb9529-480a-40a0-88cb-1b6fb0f29d02 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:45.330956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e1a76-59ac-4032-a158-652de061d929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.339041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:45.345149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 779/1561] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:07:45.346516 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a6a2f-be4d-44a7-900e-b243dd34be0e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.346869 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a6a2f-be4d-44a7-900e-b243dd34be0e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.347245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a6a2f-be4d-44a7-900e-b243dd34be0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.353083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e99e1a76-59ac-4032-a158-652de061d929 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:45.358285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e99e1a76-59ac-4032-a158-652de061d929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:45.366458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 783/1562] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:45.368063 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fed4a2-2384-446d-bbd2-e5c06774e24c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.368577 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fed4a2-2384-446d-bbd2-e5c06774e24c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.369112 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fed4a2-2384-446d-bbd2-e5c06774e24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.376205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c14a6a2f-be4d-44a7-900e-b243dd34be0e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:45.377438 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c14a6a2f-be4d-44a7-900e-b243dd34be0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:45.388966 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 780/1563] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:45.390136 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d5b0d-ed2e-4e26-9805-494d74a3189e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.390422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d5b0d-ed2e-4e26-9805-494d74a3189e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.390763 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d5b0d-ed2e-4e26-9805-494d74a3189e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.398063 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c5fed4a2-2384-446d-bbd2-e5c06774e24c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:45.399810 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c5fed4a2-2384-446d-bbd2-e5c06774e24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:45.416846 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 784/1564] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:45.417749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-bd3d5b0d-ed2e-4e26-9805-494d74a3189e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:45.418671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fed4a2-2384-446d-bbd2-e5c06774e24c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:45.419306 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-bd3d5b0d-ed2e-4e26-9805-494d74a3189e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:45.429690 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fed4a2-2384-446d-bbd2-e5c06774e24c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:45.433972 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 781/1565] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:45.434484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecabbd3-3798-4ee7-80d2-5914fb9fc3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:45.435084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d5b0d-ed2e-4e26-9805-494d74a3189e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:45.435525 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecabbd3-3798-4ee7-80d2-5914fb9fc3c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.435525 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecabbd3-3798-4ee7-80d2-5914fb9fc3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.435880 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecabbd3-3798-4ee7-80d2-5914fb9fc3c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.436599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecabbd3-3798-4ee7-80d2-5914fb9fc3c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:45.438920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecabbd3-3798-4ee7-80d2-5914fb9fc3c4 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:45.446483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d5b0d-ed2e-4e26-9805-494d74a3189e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:45.452410 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 785/1566] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:45.452410 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecabbd3-3798-4ee7-80d2-5914fb9fc3c4 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:45.457323 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f8b90-a9ac-4a59-946c-5499b96ba9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:45.457323 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f8b90-a9ac-4a59-946c-5499b96ba9a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.457323 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f8b90-a9ac-4a59-946c-5499b96ba9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.457323 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f8b90-a9ac-4a59-946c-5499b96ba9a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.457822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f8b90-a9ac-4a59-946c-5499b96ba9a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:45.457822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: WARNING py.warnings [None req-c05f8b90-a9ac-4a59-946c-5499b96ba9a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 19:07:45.457822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: warnings.warn(msg) Jul 14 19:07:45.457822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: Jul 14 19:07:45.458144 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f8b90-a9ac-4a59-946c-5499b96ba9a3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:45.463042 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecabbd3-3798-4ee7-80d2-5914fb9fc3c4 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:45.468790 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7251b6f-3aaa-483a-a354-ecbc216da409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:45.469745 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7251b6f-3aaa-483a-a354-ecbc216da409 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.469985 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7251b6f-3aaa-483a-a354-ecbc216da409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.470291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7251b6f-3aaa-483a-a354-ecbc216da409 admin admin] PATH_INFO: `/v3/projects/43a3b44b0a5f40b3955ce8027cd1cc35/users/4a5c22d1844642418e4478006c5fb1ca/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.495083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7251b6f-3aaa-483a-a354-ecbc216da409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43a3b44b0a5f40b3955ce8027cd1cc35, user_id=4a5c22d1844642418e4478006c5fb1ca, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:45.497784 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7251b6f-3aaa-483a-a354-ecbc216da409 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:45.525474 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 782/1567] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:07:45 2022] POST /identity/v3/projects => generated 387 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:45.526979 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f8b90-a9ac-4a59-946c-5499b96ba9a3 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:45.579684 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 786/1568] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:07:45 2022] PUT /identity/v3/projects/43a3b44b0a5f40b3955ce8027cd1cc35/users/4a5c22d1844642418e4478006c5fb1ca/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:07:45.580841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7251b6f-3aaa-483a-a354-ecbc216da409 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:45.609946 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f8b90-a9ac-4a59-946c-5499b96ba9a3 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:45.664863 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7251b6f-3aaa-483a-a354-ecbc216da409 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:45.669667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b755ed-5746-4445-a111-674ddeeb2a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:45.670906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b755ed-5746-4445-a111-674ddeeb2a30 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.671217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b755ed-5746-4445-a111-674ddeeb2a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.671504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b755ed-5746-4445-a111-674ddeeb2a30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.672243 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b755ed-5746-4445-a111-674ddeeb2a30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:07:45.672565 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b755ed-5746-4445-a111-674ddeeb2a30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:45.674195 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b755ed-5746-4445-a111-674ddeeb2a30 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:45.683252 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc6e44-9bd9-4a6a-8ea2-4a01ae3fa1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:45.683940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc6e44-9bd9-4a6a-8ea2-4a01ae3fa1c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.684382 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc6e44-9bd9-4a6a-8ea2-4a01ae3fa1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.684985 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc6e44-9bd9-4a6a-8ea2-4a01ae3fa1c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.685654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc6e44-9bd9-4a6a-8ea2-4a01ae3fa1c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:45.687620 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc6e44-9bd9-4a6a-8ea2-4a01ae3fa1c9 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:45.688625 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 787/1569] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:07:45.689705 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b755ed-5746-4445-a111-674ddeeb2a30 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:45.703027 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b755ed-5746-4445-a111-674ddeeb2a30 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:45.707763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200d50b-4aca-4c7c-be5b-d3ad2f5bba55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:45.708447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200d50b-4aca-4c7c-be5b-d3ad2f5bba55 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.708940 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200d50b-4aca-4c7c-be5b-d3ad2f5bba55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.709074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200d50b-4aca-4c7c-be5b-d3ad2f5bba55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.709746 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0200d50b-4aca-4c7c-be5b-d3ad2f5bba55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:07:45.710091 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0200d50b-4aca-4c7c-be5b-d3ad2f5bba55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:45.711829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0200d50b-4aca-4c7c-be5b-d3ad2f5bba55 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:45.726676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 788/1570] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:07:45.735826 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200d50b-4aca-4c7c-be5b-d3ad2f5bba55 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:45.737728 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200d50b-4aca-4c7c-be5b-d3ad2f5bba55 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:45.745247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7fbb1-d8d6-49f7-b982-c805751122a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:45.745934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7fbb1-d8d6-49f7-b982-c805751122a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.746218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7fbb1-d8d6-49f7-b982-c805751122a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.746496 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7fbb1-d8d6-49f7-b982-c805751122a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.747270 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7fbb1-d8d6-49f7-b982-c805751122a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:07:45.747583 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7fbb1-d8d6-49f7-b982-c805751122a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:45.749275 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7fbb1-d8d6-49f7-b982-c805751122a1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:45.762313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 789/1571] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:07:45.763411 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7fbb1-d8d6-49f7-b982-c805751122a1 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:45.776683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7fbb1-d8d6-49f7-b982-c805751122a1 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:45.782100 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0df6a-11f8-4d77-a8a4-2afd5a91fa2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:45.785357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0df6a-11f8-4d77-a8a4-2afd5a91fa2c admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.785357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0df6a-11f8-4d77-a8a4-2afd5a91fa2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.785357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0df6a-11f8-4d77-a8a4-2afd5a91fa2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.785357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0df6a-11f8-4d77-a8a4-2afd5a91fa2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:07:45.785357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0df6a-11f8-4d77-a8a4-2afd5a91fa2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:45.785842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 783/1572] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:07:45 2022] POST /identity/v3/projects => generated 375 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:45.785842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc6e44-9bd9-4a6a-8ea2-4a01ae3fa1c9 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:45.786011 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0df6a-11f8-4d77-a8a4-2afd5a91fa2c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:45.800811 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 790/1573] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:07:45.801661 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0df6a-11f8-4d77-a8a4-2afd5a91fa2c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:45.816810 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0df6a-11f8-4d77-a8a4-2afd5a91fa2c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:45.857673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:45.862972 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc6e44-9bd9-4a6a-8ea2-4a01ae3fa1c9 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:45.869464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d1bbe-ee67-4c05-a672-7f99837ea4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:45.870150 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d1bbe-ee67-4c05-a672-7f99837ea4dd admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.870440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d1bbe-ee67-4c05-a672-7f99837ea4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.870723 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d1bbe-ee67-4c05-a672-7f99837ea4dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.871491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d1bbe-ee67-4c05-a672-7f99837ea4dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:45.873146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d1bbe-ee67-4c05-a672-7f99837ea4dd admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:45.881337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:45.887887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 784/1574] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:45.887887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9b9e1-c35f-44dd-a8d9-c54a9ea8973b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.887887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9b9e1-c35f-44dd-a8d9-c54a9ea8973b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.887887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9b9e1-c35f-44dd-a8d9-c54a9ea8973b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.891067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd0d5b-b3e9-4584-a7d8-24ab2e67e24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:45.891067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd0d5b-b3e9-4584-a7d8-24ab2e67e24c admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.891067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd0d5b-b3e9-4584-a7d8-24ab2e67e24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.891067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd0d5b-b3e9-4584-a7d8-24ab2e67e24c admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.891474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd0d5b-b3e9-4584-a7d8-24ab2e67e24c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:45.893533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd0d5b-b3e9-4584-a7d8-24ab2e67e24c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:45.911041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-20b9b9e1-c35f-44dd-a8d9-c54a9ea8973b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:45.912362 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-20b9b9e1-c35f-44dd-a8d9-c54a9ea8973b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:45.920233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 785/1575] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:45.921150 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9b9e1-c35f-44dd-a8d9-c54a9ea8973b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:45.932857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9b9e1-c35f-44dd-a8d9-c54a9ea8973b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:45.938892 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c51e80-9cad-43b8-8c73-c0ec47ebc708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:45.939500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c51e80-9cad-43b8-8c73-c0ec47ebc708 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.939801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c51e80-9cad-43b8-8c73-c0ec47ebc708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.940073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c51e80-9cad-43b8-8c73-c0ec47ebc708 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.940784 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c51e80-9cad-43b8-8c73-c0ec47ebc708 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:45.942784 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c51e80-9cad-43b8-8c73-c0ec47ebc708 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:45.975576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 791/1576] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:07:45 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:45.976233 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd0d5b-b3e9-4584-a7d8-24ab2e67e24c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:45.986268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd0d5b-b3e9-4584-a7d8-24ab2e67e24c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:45.994482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca403a2b-cec9-4498-8230-f66fcfc315fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:45.995765 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca403a2b-cec9-4498-8230-f66fcfc315fe admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:45.996290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca403a2b-cec9-4498-8230-f66fcfc315fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:45.996804 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca403a2b-cec9-4498-8230-f66fcfc315fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:45.998020 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca403a2b-cec9-4498-8230-f66fcfc315fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:46.001501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca403a2b-cec9-4498-8230-f66fcfc315fe admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:46.050404 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 786/1577] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:07:45 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:46.051250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c51e80-9cad-43b8-8c73-c0ec47ebc708 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:46.112669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 792/1578] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:07:45 2022] POST /identity/v3/projects => generated 365 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:46.112669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca403a2b-cec9-4498-8230-f66fcfc315fe admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:46.117572 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c51e80-9cad-43b8-8c73-c0ec47ebc708 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:46.191036 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca403a2b-cec9-4498-8230-f66fcfc315fe admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:46.195648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7366e-18ba-49c5-8978-08e9c0b10418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:46.196318 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7366e-18ba-49c5-8978-08e9c0b10418 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:46.196447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5fdde-da05-4d2c-902f-7fa381795d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:46.196603 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7366e-18ba-49c5-8978-08e9c0b10418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:46.196880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7366e-18ba-49c5-8978-08e9c0b10418 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:46.196988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5fdde-da05-4d2c-902f-7fa381795d77 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:46.197276 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5fdde-da05-4d2c-902f-7fa381795d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:46.197558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5fdde-da05-4d2c-902f-7fa381795d77 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:46.197653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7366e-18ba-49c5-8978-08e9c0b10418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:46.198252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5fdde-da05-4d2c-902f-7fa381795d77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:46.199494 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7366e-18ba-49c5-8978-08e9c0b10418 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:46.200224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5fdde-da05-4d2c-902f-7fa381795d77 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:46.300684 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 793/1579] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:07:46 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:46.301516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5fdde-da05-4d2c-902f-7fa381795d77 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:46.311698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5fdde-da05-4d2c-902f-7fa381795d77 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:46.319827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 787/1580] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:07:46 2022] POST /identity/v3/projects => generated 373 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:46.319827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7366e-18ba-49c5-8978-08e9c0b10418 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:46.320246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d912e-6976-49cc-881d-354ff8c4f045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:46.320246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d912e-6976-49cc-881d-354ff8c4f045 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:46.320246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d912e-6976-49cc-881d-354ff8c4f045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:46.320681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d912e-6976-49cc-881d-354ff8c4f045 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:46.322336 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d912e-6976-49cc-881d-354ff8c4f045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:46.324615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d912e-6976-49cc-881d-354ff8c4f045 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:46.384816 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:46.390029 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7366e-18ba-49c5-8978-08e9c0b10418 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:46.394747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef149381-bcd2-4443-996c-805dfd7b814e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:46.395673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef149381-bcd2-4443-996c-805dfd7b814e admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:46.395959 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef149381-bcd2-4443-996c-805dfd7b814e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:46.396264 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef149381-bcd2-4443-996c-805dfd7b814e admin admin] PATH_INFO: `/v3/projects/01bc27a5fcbb4d9d94cab5f5906c43fa/users/5aa7ff2fa5454bba9268e89c44b75de6/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:46.404768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 794/1581] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:07:46 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:46.405698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d912e-6976-49cc-881d-354ff8c4f045 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:46.413476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef149381-bcd2-4443-996c-805dfd7b814e admin admin] RBAC: Authorizing `identity:create_grant(project_id=01bc27a5fcbb4d9d94cab5f5906c43fa, user_id=5aa7ff2fa5454bba9268e89c44b75de6, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:46.414808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: WARNING py.warnings [None req-ef149381-bcd2-4443-996c-805dfd7b814e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 19:07:46.414808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: warnings.warn(msg) Jul 14 19:07:46.414808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: Jul 14 19:07:46.416949 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef149381-bcd2-4443-996c-805dfd7b814e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:46.463050 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d912e-6976-49cc-881d-354ff8c4f045 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:46.466932 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958da0c4-b9b5-49ba-9832-05e04922b11d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:46.467690 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958da0c4-b9b5-49ba-9832-05e04922b11d service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:46.468759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958da0c4-b9b5-49ba-9832-05e04922b11d service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:46.468759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958da0c4-b9b5-49ba-9832-05e04922b11d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:46.495190 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 788/1582] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:07:46 2022] PUT /identity/v3/projects/01bc27a5fcbb4d9d94cab5f5906c43fa/users/5aa7ff2fa5454bba9268e89c44b75de6/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:07:46.496140 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef149381-bcd2-4443-996c-805dfd7b814e admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:46.517747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:46.535454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958da0c4-b9b5-49ba-9832-05e04922b11d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:46.536914 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958da0c4-b9b5-49ba-9832-05e04922b11d service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:46.547485 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 795/1583] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:07:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:07:46.548413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958da0c4-b9b5-49ba-9832-05e04922b11d service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:46.558182 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958da0c4-b9b5-49ba-9832-05e04922b11d service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:46.565112 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccb987-f7fd-44a0-9085-f6e93a9425a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:46.565811 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccb987-f7fd-44a0-9085-f6e93a9425a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:46.566136 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccb987-f7fd-44a0-9085-f6e93a9425a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:46.566416 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccb987-f7fd-44a0-9085-f6e93a9425a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:46.567119 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ccb987-f7fd-44a0-9085-f6e93a9425a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:46.572088 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ccb987-f7fd-44a0-9085-f6e93a9425a5 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:46.572284 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef149381-bcd2-4443-996c-805dfd7b814e admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:46.574413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34265d2d-cad2-4f29-a70a-05fb129fd8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:46.575309 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34265d2d-cad2-4f29-a70a-05fb129fd8cb admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:46.575601 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34265d2d-cad2-4f29-a70a-05fb129fd8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:46.575881 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34265d2d-cad2-4f29-a70a-05fb129fd8cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:46.576649 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34265d2d-cad2-4f29-a70a-05fb129fd8cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:46.578344 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34265d2d-cad2-4f29-a70a-05fb129fd8cb admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:46.590572 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 789/1584] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:46.595036 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34265d2d-cad2-4f29-a70a-05fb129fd8cb admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:46.601852 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34265d2d-cad2-4f29-a70a-05fb129fd8cb admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:46.606641 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8190dd-15c1-4f3d-85e7-90bc81fde4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:46.607887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8190dd-15c1-4f3d-85e7-90bc81fde4ae admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:46.608176 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8190dd-15c1-4f3d-85e7-90bc81fde4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:46.608459 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8190dd-15c1-4f3d-85e7-90bc81fde4ae admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:46.609142 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8190dd-15c1-4f3d-85e7-90bc81fde4ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:46.611071 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8190dd-15c1-4f3d-85e7-90bc81fde4ae admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:46.707912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 796/1585] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:07:46 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:46.711118 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 790/1586] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:07:46 2022] POST /identity/v3/users => generated 283 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:46.711118 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8190dd-15c1-4f3d-85e7-90bc81fde4ae admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:46.712450 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccb987-f7fd-44a0-9085-f6e93a9425a5 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:46.722936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8190dd-15c1-4f3d-85e7-90bc81fde4ae admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:46.723456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccb987-f7fd-44a0-9085-f6e93a9425a5 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:46.728498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61152e67-739b-4c28-a1ce-1ee02289d08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:46.729499 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25f845-a1b2-42ac-a6df-b3c025cec311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:46.730424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61152e67-739b-4c28-a1ce-1ee02289d08c admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:46.730424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61152e67-739b-4c28-a1ce-1ee02289d08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:46.730706 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25f845-a1b2-42ac-a6df-b3c025cec311 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:46.730798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61152e67-739b-4c28-a1ce-1ee02289d08c admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:46.730903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25f845-a1b2-42ac-a6df-b3c025cec311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:46.730903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25f845-a1b2-42ac-a6df-b3c025cec311 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:46.731324 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61152e67-739b-4c28-a1ce-1ee02289d08c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:46.731530 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25f845-a1b2-42ac-a6df-b3c025cec311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:46.733338 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61152e67-739b-4c28-a1ce-1ee02289d08c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:46.733887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25f845-a1b2-42ac-a6df-b3c025cec311 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:46.745477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 791/1587] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:46.746214 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61152e67-739b-4c28-a1ce-1ee02289d08c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:46.756789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61152e67-739b-4c28-a1ce-1ee02289d08c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:46.762140 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780a368-57f0-40ac-9521-9aa85c7641e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:46.762862 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780a368-57f0-40ac-9521-9aa85c7641e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:46.763147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780a368-57f0-40ac-9521-9aa85c7641e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:46.763488 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780a368-57f0-40ac-9521-9aa85c7641e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:46.764699 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780a368-57f0-40ac-9521-9aa85c7641e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:46.766742 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780a368-57f0-40ac-9521-9aa85c7641e7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:46.791929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 797/1588] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:07:46 2022] POST /identity/v3/users => generated 287 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:46.793244 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e090d0-f808-450b-ac3f-b9f0ea5e0af4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:46.793545 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e090d0-f808-450b-ac3f-b9f0ea5e0af4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:46.793816 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e090d0-f808-450b-ac3f-b9f0ea5e0af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:46.811955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-97e090d0-f808-450b-ac3f-b9f0ea5e0af4 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:46.813177 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-97e090d0-f808-450b-ac3f-b9f0ea5e0af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:46.822104 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 798/1589] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:46.823684 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e090d0-f808-450b-ac3f-b9f0ea5e0af4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:46.852758 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 792/1590] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:07:46 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:46.852758 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780a368-57f0-40ac-9521-9aa85c7641e7 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:46.852960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e090d0-f808-450b-ac3f-b9f0ea5e0af4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:46.856720 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c2b71-7f7d-420a-b2b8-2e2626863e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:46.858732 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c2b71-7f7d-420a-b2b8-2e2626863e99 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:46.858886 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780a368-57f0-40ac-9521-9aa85c7641e7 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:46.859133 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c2b71-7f7d-420a-b2b8-2e2626863e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:46.859325 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c2b71-7f7d-420a-b2b8-2e2626863e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:46.860041 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c2b71-7f7d-420a-b2b8-2e2626863e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:46.862277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c2b71-7f7d-420a-b2b8-2e2626863e99 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:46.864079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503039e-667a-4fed-b46e-1ae1e21cd5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:46.865169 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503039e-667a-4fed-b46e-1ae1e21cd5df admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:46.865538 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503039e-667a-4fed-b46e-1ae1e21cd5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:46.866012 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503039e-667a-4fed-b46e-1ae1e21cd5df admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:46.867131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503039e-667a-4fed-b46e-1ae1e21cd5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:46.869015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503039e-667a-4fed-b46e-1ae1e21cd5df admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:46.877134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 799/1591] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:46.877134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c2b71-7f7d-420a-b2b8-2e2626863e99 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:46.891102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 793/1592] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:46.892391 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503039e-667a-4fed-b46e-1ae1e21cd5df admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:46.892564 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c2b71-7f7d-420a-b2b8-2e2626863e99 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:46.900762 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c9390-89d1-4248-bf3d-9bdfe9825efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:46.901663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c9390-89d1-4248-bf3d-9bdfe9825efe admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:46.902304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c9390-89d1-4248-bf3d-9bdfe9825efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:46.902433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c9390-89d1-4248-bf3d-9bdfe9825efe admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:46.903559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c9390-89d1-4248-bf3d-9bdfe9825efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:46.905926 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c9390-89d1-4248-bf3d-9bdfe9825efe admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:46.906527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503039e-667a-4fed-b46e-1ae1e21cd5df admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:46.915430 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4efe0-a27b-42ab-8c46-d699cc4d28b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:46.922640 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4efe0-a27b-42ab-8c46-d699cc4d28b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:46.922640 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4efe0-a27b-42ab-8c46-d699cc4d28b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:46.922640 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4efe0-a27b-42ab-8c46-d699cc4d28b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:46.922640 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4efe0-a27b-42ab-8c46-d699cc4d28b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:46.922977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 800/1593] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:46.922977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c9390-89d1-4248-bf3d-9bdfe9825efe admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:46.923131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4efe0-a27b-42ab-8c46-d699cc4d28b4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:46.938345 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c9390-89d1-4248-bf3d-9bdfe9825efe admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:46.943413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 794/1594] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:46.944106 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94352202-b131-4e33-80ac-4b7877bb91da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:46.944838 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4efe0-a27b-42ab-8c46-d699cc4d28b4 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:46.949484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94352202-b131-4e33-80ac-4b7877bb91da admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:46.949484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94352202-b131-4e33-80ac-4b7877bb91da admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:46.949484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94352202-b131-4e33-80ac-4b7877bb91da admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:46.949484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94352202-b131-4e33-80ac-4b7877bb91da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:46.950171 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-34a4efe0-a27b-42ab-8c46-d699cc4d28b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:46.950285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94352202-b131-4e33-80ac-4b7877bb91da admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:46.960826 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 801/1595] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:46.961682 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94352202-b131-4e33-80ac-4b7877bb91da admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:46.972281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94352202-b131-4e33-80ac-4b7877bb91da admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:46.977611 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82908a15-7ed3-4c93-973d-25d8fe1faab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:46.978192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82908a15-7ed3-4c93-973d-25d8fe1faab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:46.978472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82908a15-7ed3-4c93-973d-25d8fe1faab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:46.978908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82908a15-7ed3-4c93-973d-25d8fe1faab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:46.979685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82908a15-7ed3-4c93-973d-25d8fe1faab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:46.981295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82908a15-7ed3-4c93-973d-25d8fe1faab9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:46.992287 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 802/1596] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:46.996009 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82908a15-7ed3-4c93-973d-25d8fe1faab9 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:47.004878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82908a15-7ed3-4c93-973d-25d8fe1faab9 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:47.010383 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7e364-239c-47b5-9d12-76d54b5ff197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:47.010960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7e364-239c-47b5-9d12-76d54b5ff197 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:47.011384 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7e364-239c-47b5-9d12-76d54b5ff197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:47.011562 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7e364-239c-47b5-9d12-76d54b5ff197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:47.012360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7e364-239c-47b5-9d12-76d54b5ff197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:47.014711 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7e364-239c-47b5-9d12-76d54b5ff197 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:47.025533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 803/1597] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:47.026360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7e364-239c-47b5-9d12-76d54b5ff197 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:47.038322 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4efe0-a27b-42ab-8c46-d699cc4d28b4 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:47.039596 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7e364-239c-47b5-9d12-76d54b5ff197 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:47.042479 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c161de-5cd9-4231-ad9e-8be6fa590210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:47.043139 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c161de-5cd9-4231-ad9e-8be6fa590210 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:47.043530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c161de-5cd9-4231-ad9e-8be6fa590210 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:47.043762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c161de-5cd9-4231-ad9e-8be6fa590210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:47.044789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca97ca8-52d0-4727-926a-bb9aa3ea34cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:47.045799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca97ca8-52d0-4727-926a-bb9aa3ea34cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:47.046088 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca97ca8-52d0-4727-926a-bb9aa3ea34cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:47.046419 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca97ca8-52d0-4727-926a-bb9aa3ea34cd admin admin] PATH_INFO: `/v3/projects/5eb090b54f744ce5bfd44a2fa18644b7/users/b72067d66a7649e7b8c120dd4d3fe635/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:47.054959 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c161de-5cd9-4231-ad9e-8be6fa590210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:47.056451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c161de-5cd9-4231-ad9e-8be6fa590210 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:47.067112 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca97ca8-52d0-4727-926a-bb9aa3ea34cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eb090b54f744ce5bfd44a2fa18644b7, user_id=b72067d66a7649e7b8c120dd4d3fe635, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:47.069833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 795/1598] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:07:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:07:47.070014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca97ca8-52d0-4727-926a-bb9aa3ea34cd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:47.071023 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c161de-5cd9-4231-ad9e-8be6fa590210 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:47.083026 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c161de-5cd9-4231-ad9e-8be6fa590210 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:47.090289 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf729cfe-38ee-4980-9ec3-46b06ec371a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:47.091690 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf729cfe-38ee-4980-9ec3-46b06ec371a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:47.094285 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf729cfe-38ee-4980-9ec3-46b06ec371a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:47.094635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf729cfe-38ee-4980-9ec3-46b06ec371a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:47.095805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf729cfe-38ee-4980-9ec3-46b06ec371a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:47.098298 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf729cfe-38ee-4980-9ec3-46b06ec371a3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:47.101572 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:47.110664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 796/1599] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:47.111764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf729cfe-38ee-4980-9ec3-46b06ec371a3 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:47.121850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf729cfe-38ee-4980-9ec3-46b06ec371a3 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:47.127446 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da859705-24a9-4792-8b6c-6c16f748b58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:47.128355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da859705-24a9-4792-8b6c-6c16f748b58a admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:47.128649 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da859705-24a9-4792-8b6c-6c16f748b58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:47.128929 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da859705-24a9-4792-8b6c-6c16f748b58a admin admin] PATH_INFO: `/v3/projects/721bf97931464b0f99e6379267379d32/users/0cefc934688f4f618c86b882f7447297/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:47.150202 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da859705-24a9-4792-8b6c-6c16f748b58a admin admin] RBAC: Authorizing `identity:create_grant(project_id=721bf97931464b0f99e6379267379d32, user_id=0cefc934688f4f618c86b882f7447297, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:47.160129 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da859705-24a9-4792-8b6c-6c16f748b58a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:47.160290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 804/1600] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:07:47 2022] PUT /identity/v3/projects/5eb090b54f744ce5bfd44a2fa18644b7/users/b72067d66a7649e7b8c120dd4d3fe635/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:07:47.160290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca97ca8-52d0-4727-926a-bb9aa3ea34cd admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:47.243433 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 797/1601] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:07:47 2022] PUT /identity/v3/projects/721bf97931464b0f99e6379267379d32/users/0cefc934688f4f618c86b882f7447297/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:07:47.245509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da859705-24a9-4792-8b6c-6c16f748b58a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:47.266643 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca97ca8-52d0-4727-926a-bb9aa3ea34cd admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:47.295498 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:47.342676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99429d6a-aa84-4de8-b8c7-b417c32d6e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:47.346258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99429d6a-aa84-4de8-b8c7-b417c32d6e3e admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:47.346258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99429d6a-aa84-4de8-b8c7-b417c32d6e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:47.346258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99429d6a-aa84-4de8-b8c7-b417c32d6e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:47.346258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99429d6a-aa84-4de8-b8c7-b417c32d6e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:47.346765 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99429d6a-aa84-4de8-b8c7-b417c32d6e3e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:47.358982 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 805/1602] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:47.359849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99429d6a-aa84-4de8-b8c7-b417c32d6e3e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:47.368037 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da859705-24a9-4792-8b6c-6c16f748b58a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:47.374550 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99429d6a-aa84-4de8-b8c7-b417c32d6e3e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:47.375705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33167693-c798-491b-84b5-385fd863f7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:47.376604 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33167693-c798-491b-84b5-385fd863f7ce admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:47.377054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33167693-c798-491b-84b5-385fd863f7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:47.377523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33167693-c798-491b-84b5-385fd863f7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:47.378449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33167693-c798-491b-84b5-385fd863f7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:47.379963 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d32044-2a5a-4136-9b2e-ca6cdb7f4eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:47.380466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33167693-c798-491b-84b5-385fd863f7ce admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:47.380909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d32044-2a5a-4136-9b2e-ca6cdb7f4eb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:47.381202 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d32044-2a5a-4136-9b2e-ca6cdb7f4eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:47.381492 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d32044-2a5a-4136-9b2e-ca6cdb7f4eb0 admin admin] PATH_INFO: `/v3/projects/9f089badaa0b48988c62a394a7a4b179/users/934dc97df7434ace9c19757a9cd8cf18/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:47.393950 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 798/1603] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:47.394796 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33167693-c798-491b-84b5-385fd863f7ce admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:47.406221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d32044-2a5a-4136-9b2e-ca6cdb7f4eb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f089badaa0b48988c62a394a7a4b179, user_id=934dc97df7434ace9c19757a9cd8cf18, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:47.410807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d32044-2a5a-4136-9b2e-ca6cdb7f4eb0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:47.410952 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33167693-c798-491b-84b5-385fd863f7ce admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:47.417716 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f340bd7-12f7-4580-bdec-1221e57316e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:47.419087 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f340bd7-12f7-4580-bdec-1221e57316e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:47.419563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f340bd7-12f7-4580-bdec-1221e57316e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:47.420538 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f340bd7-12f7-4580-bdec-1221e57316e0 admin admin] PATH_INFO: `/v3/projects/1050d9df0b3a4888bc6c1a7299fd4b7a/users/9d1d7a0d45b040f2a927b299638d3c44/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:47.444904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f340bd7-12f7-4580-bdec-1221e57316e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1050d9df0b3a4888bc6c1a7299fd4b7a, user_id=9d1d7a0d45b040f2a927b299638d3c44, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:47.448789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f340bd7-12f7-4580-bdec-1221e57316e0 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:47.515640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 806/1604] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:07:47 2022] PUT /identity/v3/projects/9f089badaa0b48988c62a394a7a4b179/users/934dc97df7434ace9c19757a9cd8cf18/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:07:47.515750 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074d5c5-d33d-4476-9f35-1da3c094e7ff None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:47.516190 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074d5c5-d33d-4476-9f35-1da3c094e7ff None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:47.516560 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074d5c5-d33d-4476-9f35-1da3c094e7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:47.536877 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7074d5c5-d33d-4476-9f35-1da3c094e7ff None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:47.538229 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7074d5c5-d33d-4476-9f35-1da3c094e7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:47.555236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 799/1605] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:07:47 2022] PUT /identity/v3/projects/1050d9df0b3a4888bc6c1a7299fd4b7a/users/9d1d7a0d45b040f2a927b299638d3c44/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:07:47.555236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f4edb-5266-44bf-a63b-f6756cad242d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:47.555236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f4edb-5266-44bf-a63b-f6756cad242d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:47.555236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f4edb-5266-44bf-a63b-f6756cad242d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:47.597261 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0f0f4edb-5266-44bf-a63b-f6756cad242d None None] MFA Rules not processed for user `0cefc934688f4f618c86b882f7447297`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:47.599249 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0f0f4edb-5266-44bf-a63b-f6756cad242d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:47.633717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 807/1606] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:47.635284 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074d5c5-d33d-4476-9f35-1da3c094e7ff None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:47.647853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074d5c5-d33d-4476-9f35-1da3c094e7ff None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:47.655821 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006b1f4-afd5-4cc0-86cb-c1df5e879a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:47.657327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006b1f4-afd5-4cc0-86cb-c1df5e879a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:47.657776 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006b1f4-afd5-4cc0-86cb-c1df5e879a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:47.658529 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006b1f4-afd5-4cc0-86cb-c1df5e879a27 admin admin] PATH_INFO: `/v3/projects/8b3c6f140ddd467484b2d73f4ba2fc41/users/d74fa52ecabd4fa0b0a31f0e2de5d167/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:47.684618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006b1f4-afd5-4cc0-86cb-c1df5e879a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b3c6f140ddd467484b2d73f4ba2fc41, user_id=d74fa52ecabd4fa0b0a31f0e2de5d167, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:47.688834 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006b1f4-afd5-4cc0-86cb-c1df5e879a27 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:47.718753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 800/1607] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:47.719961 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f4edb-5266-44bf-a63b-f6756cad242d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:47.746230 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f4edb-5266-44bf-a63b-f6756cad242d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:47.754680 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bc732-e0c1-41bd-b955-3ce6bb77c072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:47.757372 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bc732-e0c1-41bd-b955-3ce6bb77c072 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:47.757824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bc732-e0c1-41bd-b955-3ce6bb77c072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:47.758268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bc732-e0c1-41bd-b955-3ce6bb77c072 admin admin] PATH_INFO: `/v3/projects/aa66aa59ee324394a45b3059ae1f49b5/users/7d5fa180ee554e8b8e46321892969489/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:47.770309 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 808/1608] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:07:47 2022] PUT /identity/v3/projects/8b3c6f140ddd467484b2d73f4ba2fc41/users/d74fa52ecabd4fa0b0a31f0e2de5d167/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:07:47.772406 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de492b-2b3b-44d6-8bdf-c87a01e2bf07 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:47.772881 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de492b-2b3b-44d6-8bdf-c87a01e2bf07 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:47.773386 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de492b-2b3b-44d6-8bdf-c87a01e2bf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:47.782672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bc732-e0c1-41bd-b955-3ce6bb77c072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa66aa59ee324394a45b3059ae1f49b5, user_id=7d5fa180ee554e8b8e46321892969489, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:47.786921 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bc732-e0c1-41bd-b955-3ce6bb77c072 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:47.818691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-a8de492b-2b3b-44d6-8bdf-c87a01e2bf07 None None] MFA Rules not processed for user `934dc97df7434ace9c19757a9cd8cf18`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:47.820670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a8de492b-2b3b-44d6-8bdf-c87a01e2bf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:47.873958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 801/1609] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:07:47 2022] PUT /identity/v3/projects/aa66aa59ee324394a45b3059ae1f49b5/users/7d5fa180ee554e8b8e46321892969489/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:07:47.873958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc00fc5-b4b8-4e23-87dd-cff0609bf4f0 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:47.873958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc00fc5-b4b8-4e23-87dd-cff0609bf4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:47.873958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc00fc5-b4b8-4e23-87dd-cff0609bf4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:47.925190 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 809/1610] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:47.926215 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de492b-2b3b-44d6-8bdf-c87a01e2bf07 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:47.931051 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:47.941145 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:47.944614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ecc00fc5-b4b8-4e23-87dd-cff0609bf4f0 None None] MFA Rules not processed for user `9d1d7a0d45b040f2a927b299638d3c44`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:47.946918 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ecc00fc5-b4b8-4e23-87dd-cff0609bf4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:48.005199 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de492b-2b3b-44d6-8bdf-c87a01e2bf07 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:48.089210 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52147ea-46b3-49a0-8e4f-a055170567bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:48.090013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52147ea-46b3-49a0-8e4f-a055170567bc service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:48.092733 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52147ea-46b3-49a0-8e4f-a055170567bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:48.093671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52147ea-46b3-49a0-8e4f-a055170567bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:48.110394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 802/1611] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:48.112054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9c80c-9006-4262-a7e3-d57f623d2baf None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:48.112349 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9c80c-9006-4262-a7e3-d57f623d2baf None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:48.112627 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9c80c-9006-4262-a7e3-d57f623d2baf None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:48.120074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 803/1612] 10.222.0.69 () {54 vars in 961 bytes} [Thu Jul 14 19:07:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:07:48.121520 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58ab28-1a01-41d6-86d2-8f5f507f98aa None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:48.121817 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58ab28-1a01-41d6-86d2-8f5f507f98aa None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:48.123262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58ab28-1a01-41d6-86d2-8f5f507f98aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:48.157460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-aa58ab28-1a01-41d6-86d2-8f5f507f98aa None None] MFA Rules not processed for user `d74fa52ecabd4fa0b0a31f0e2de5d167`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:48.160152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-aa58ab28-1a01-41d6-86d2-8f5f507f98aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:48.201801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52147ea-46b3-49a0-8e4f-a055170567bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:48.203716 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52147ea-46b3-49a0-8e4f-a055170567bc service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:48.222980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 810/1613] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:07:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:07:48.225597 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d20465-c698-4abf-aa9d-93e9d3077628 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:48.226670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d20465-c698-4abf-aa9d-93e9d3077628 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:48.227283 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d20465-c698-4abf-aa9d-93e9d3077628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:48.250321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-56d20465-c698-4abf-aa9d-93e9d3077628 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:48.252435 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-56d20465-c698-4abf-aa9d-93e9d3077628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:48.263398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 811/1614] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:48.265408 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d20465-c698-4abf-aa9d-93e9d3077628 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:48.314707 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 804/1615] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:48.316170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a3086-d633-4cdb-afb6-d706d2b6a6c7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:48.316537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a3086-d633-4cdb-afb6-d706d2b6a6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:48.318018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a3086-d633-4cdb-afb6-d706d2b6a6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:48.356896 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d20465-c698-4abf-aa9d-93e9d3077628 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:48.361436 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbaf98d-88c8-432f-9bf4-3d67719e71d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:48.362159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbaf98d-88c8-432f-9bf4-3d67719e71d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:48.364194 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbaf98d-88c8-432f-9bf4-3d67719e71d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:48.364636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbaf98d-88c8-432f-9bf4-3d67719e71d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:48.367711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-487a3086-d633-4cdb-afb6-d706d2b6a6c7 None None] MFA Rules not processed for user `8d27d6df5cb64697adebd961dfbec324`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:48.371980 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-487a3086-d633-4cdb-afb6-d706d2b6a6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:48.453079 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbaf98d-88c8-432f-9bf4-3d67719e71d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:48.455394 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbaf98d-88c8-432f-9bf4-3d67719e71d4 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:48.466585 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 812/1616] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:07:48 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:07:48.475415 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509ba63-4f39-4e5a-80ef-da27b645eace None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:48.475415 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509ba63-4f39-4e5a-80ef-da27b645eace None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:48.475786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509ba63-4f39-4e5a-80ef-da27b645eace None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:48.487503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 813/1617] 10.222.0.69 () {54 vars in 961 bytes} [Thu Jul 14 19:07:48 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:07:48.489584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509ba63-4f39-4e5a-80ef-da27b645eace None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:48.507647 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 805/1618] 10.222.0.69 () {58 vars in 1058 bytes} [Thu Jul 14 19:07:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:48.508569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a3086-d633-4cdb-afb6-d706d2b6a6c7 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:48.522604 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a3086-d633-4cdb-afb6-d706d2b6a6c7 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:48.529077 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66259aba-0f4d-4c39-8a74-93d84ccff320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:48.529789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66259aba-0f4d-4c39-8a74-93d84ccff320 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:48.530080 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66259aba-0f4d-4c39-8a74-93d84ccff320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:48.530361 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66259aba-0f4d-4c39-8a74-93d84ccff320 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:48.531102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66259aba-0f4d-4c39-8a74-93d84ccff320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:48.531554 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509ba63-4f39-4e5a-80ef-da27b645eace None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:48.533468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66259aba-0f4d-4c39-8a74-93d84ccff320 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:48.540605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:48.546074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b361004-4ee4-4cb0-a877-bd68b44c1135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:48.547149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b361004-4ee4-4cb0-a877-bd68b44c1135 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:48.547652 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b361004-4ee4-4cb0-a877-bd68b44c1135 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:48.548122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b361004-4ee4-4cb0-a877-bd68b44c1135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:48.572106 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b361004-4ee4-4cb0-a877-bd68b44c1135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:48.575683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b361004-4ee4-4cb0-a877-bd68b44c1135 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:48.590857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 814/1619] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:07:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:07:48.593530 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ddaeab-95a4-4764-8359-eec5a9518620 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:48.594116 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ddaeab-95a4-4764-8359-eec5a9518620 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:48.594606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ddaeab-95a4-4764-8359-eec5a9518620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:48.640066 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-16ddaeab-95a4-4764-8359-eec5a9518620 None None] MFA Rules not processed for user `8d27d6df5cb64697adebd961dfbec324`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:48.643314 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-16ddaeab-95a4-4764-8359-eec5a9518620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:48.659823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 815/1620] 10.222.0.69 () {58 vars in 1058 bytes} [Thu Jul 14 19:07:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:48.660917 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ddaeab-95a4-4764-8359-eec5a9518620 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:48.674964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ddaeab-95a4-4764-8359-eec5a9518620 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:48.682832 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159afeb-65e7-439e-9015-191595990fb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:48.683669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159afeb-65e7-439e-9015-191595990fb9 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:48.684006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159afeb-65e7-439e-9015-191595990fb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:48.684359 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159afeb-65e7-439e-9015-191595990fb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:48.692720 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 806/1621] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:07:48 2022] POST /identity/v3/projects => generated 371 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:48.698082 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66259aba-0f4d-4c39-8a74-93d84ccff320 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:48.793760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159afeb-65e7-439e-9015-191595990fb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:48.796019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159afeb-65e7-439e-9015-191595990fb9 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:48.806413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 816/1622] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:07:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:07:48.807287 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159afeb-65e7-439e-9015-191595990fb9 service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:48.819745 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:48.825162 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159afeb-65e7-439e-9015-191595990fb9 service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:48.906642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82f7c9-4140-44b4-9e31-117733c568bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:48.909455 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82f7c9-4140-44b4-9e31-117733c568bb admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:48.910643 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82f7c9-4140-44b4-9e31-117733c568bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:48.911540 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82f7c9-4140-44b4-9e31-117733c568bb admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:48.915035 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf82f7c9-4140-44b4-9e31-117733c568bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:48.922593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf82f7c9-4140-44b4-9e31-117733c568bb admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:48.938297 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66259aba-0f4d-4c39-8a74-93d84ccff320 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:48.951232 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb4207-ca4f-4224-8668-fdbe6f9f81e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:48.952194 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb4207-ca4f-4224-8668-fdbe6f9f81e2 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:48.952609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb4207-ca4f-4224-8668-fdbe6f9f81e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:48.953114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb4207-ca4f-4224-8668-fdbe6f9f81e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:49.036230 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 817/1623] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:07:48 2022] POST /identity/v3/users => generated 292 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:49.039466 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82f7c9-4140-44b4-9e31-117733c568bb admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:49.065122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82f7c9-4140-44b4-9e31-117733c568bb admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:49.078671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12575e34-a270-453f-aa6c-d16951e86e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:49.080693 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12575e34-a270-453f-aa6c-d16951e86e0e admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:49.081785 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12575e34-a270-453f-aa6c-d16951e86e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:49.082480 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12575e34-a270-453f-aa6c-d16951e86e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:49.085054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12575e34-a270-453f-aa6c-d16951e86e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:49.090289 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12575e34-a270-453f-aa6c-d16951e86e0e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:49.116113 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 818/1624] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:49.121220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12575e34-a270-453f-aa6c-d16951e86e0e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:49.124043 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb4207-ca4f-4224-8668-fdbe6f9f81e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:49.126089 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb4207-ca4f-4224-8668-fdbe6f9f81e2 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:49.143137 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12575e34-a270-453f-aa6c-d16951e86e0e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:49.151679 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b079e9-ee54-46b0-b17e-060e070d4bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:49.151980 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 807/1625] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:07:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:07:49.153429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b079e9-ee54-46b0-b17e-060e070d4bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:49.154029 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b079e9-ee54-46b0-b17e-060e070d4bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:49.155204 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b079e9-ee54-46b0-b17e-060e070d4bd5 admin admin] PATH_INFO: `/v3/projects/3b1a80c28e164a6c834d0043ccf5d2d0/users/288c7fdaaba242b68a1b4b649837f5c4/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:49.205063 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:49.219808 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b079e9-ee54-46b0-b17e-060e070d4bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b1a80c28e164a6c834d0043ccf5d2d0, user_id=288c7fdaaba242b68a1b4b649837f5c4, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:49.235426 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b079e9-ee54-46b0-b17e-060e070d4bd5 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:49.361854 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 819/1626] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:07:49 2022] PUT /identity/v3/projects/3b1a80c28e164a6c834d0043ccf5d2d0/users/288c7fdaaba242b68a1b4b649837f5c4/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:07:49.363264 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb4207-ca4f-4224-8668-fdbe6f9f81e2 service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:49.377451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb4207-ca4f-4224-8668-fdbe6f9f81e2 service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:49.454792 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9094c-c15b-40f4-8e8f-2a79023c4393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:49.461332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9094c-c15b-40f4-8e8f-2a79023c4393 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:49.461332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9094c-c15b-40f4-8e8f-2a79023c4393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:49.461332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9094c-c15b-40f4-8e8f-2a79023c4393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:49.461332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9094c-c15b-40f4-8e8f-2a79023c4393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:49.461332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9094c-c15b-40f4-8e8f-2a79023c4393 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:49.473354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 808/1627] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:07:49.478237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b079e9-ee54-46b0-b17e-060e070d4bd5 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:49.491974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b079e9-ee54-46b0-b17e-060e070d4bd5 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:49.504074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598932ee-b310-4e46-8599-10210db30fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:49.506433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598932ee-b310-4e46-8599-10210db30fb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:49.506747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598932ee-b310-4e46-8599-10210db30fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:49.507729 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598932ee-b310-4e46-8599-10210db30fb6 admin admin] PATH_INFO: `/v3/projects/3b1a80c28e164a6c834d0043ccf5d2d0/users/288c7fdaaba242b68a1b4b649837f5c4/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:49.517309 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598932ee-b310-4e46-8599-10210db30fb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b1a80c28e164a6c834d0043ccf5d2d0, user_id=288c7fdaaba242b68a1b4b649837f5c4, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:49.520012 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc75f6-f737-42f2-a3ef-ffab8f67c32b None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:49.520533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598932ee-b310-4e46-8599-10210db30fb6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:49.521013 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc75f6-f737-42f2-a3ef-ffab8f67c32b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:49.521510 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc75f6-f737-42f2-a3ef-ffab8f67c32b None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:49.523793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 809/1628] 10.222.0.69 () {54 vars in 912 bytes} [Thu Jul 14 19:07:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:07:49.531700 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e46ec-a758-440a-b532-c0b67834fb51 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:49.532286 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e46ec-a758-440a-b532-c0b67834fb51 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:49.532786 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e46ec-a758-440a-b532-c0b67834fb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:49.547842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 820/1629] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:07:49 2022] PUT /identity/v3/projects/3b1a80c28e164a6c834d0043ccf5d2d0/users/288c7fdaaba242b68a1b4b649837f5c4/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:07:49.559105 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57774d7f-6d5e-4137-bc1f-7cdb5b33f6d7 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:49.559105 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57774d7f-6d5e-4137-bc1f-7cdb5b33f6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:49.559105 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57774d7f-6d5e-4137-bc1f-7cdb5b33f6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:49.586241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-de3e46ec-a758-440a-b532-c0b67834fb51 None None] MFA Rules not processed for user `8d27d6df5cb64697adebd961dfbec324`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:49.595242 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-de3e46ec-a758-440a-b532-c0b67834fb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:49.619768 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:49.630833 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-57774d7f-6d5e-4137-bc1f-7cdb5b33f6d7 None None] MFA Rules not processed for user `288c7fdaaba242b68a1b4b649837f5c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:49.634593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-57774d7f-6d5e-4137-bc1f-7cdb5b33f6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:49.753243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 810/1630] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 14 19:07:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:49.775092 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e46ec-a758-440a-b532-c0b67834fb51 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:49.825655 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 821/1631] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:49 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:07:49.911798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e46ec-a758-440a-b532-c0b67834fb51 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:49.917104 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a26f258-cdf8-4f47-a66d-2f22ebb726e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:49.917955 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26f258-cdf8-4f47-a66d-2f22ebb726e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:49.918320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26f258-cdf8-4f47-a66d-2f22ebb726e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:49.918677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26f258-cdf8-4f47-a66d-2f22ebb726e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:49.943534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a26f258-cdf8-4f47-a66d-2f22ebb726e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:49.947217 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a26f258-cdf8-4f47-a66d-2f22ebb726e7 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:49.953612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea95c2-a358-4fd3-b39a-1427bdb0adc5 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:49.954108 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea95c2-a358-4fd3-b39a-1427bdb0adc5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:49.954473 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea95c2-a358-4fd3-b39a-1427bdb0adc5 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:49.956308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 822/1632] 10.222.0.69 () {54 vars in 912 bytes} [Thu Jul 14 19:07:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 19:07:49.966358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e15ec9-464c-4d83-b0f9-0d349135ef76 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:49.966908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e15ec9-464c-4d83-b0f9-0d349135ef76 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:49.967342 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e15ec9-464c-4d83-b0f9-0d349135ef76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:49.982437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 811/1633] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:07:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:07:50.022578 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-02e15ec9-464c-4d83-b0f9-0d349135ef76 None None] MFA Rules not processed for user `8d27d6df5cb64697adebd961dfbec324`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:50.028567 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-02e15ec9-464c-4d83-b0f9-0d349135ef76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:50.030058 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a26f258-cdf8-4f47-a66d-2f22ebb726e7 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:50.041627 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a26f258-cdf8-4f47-a66d-2f22ebb726e7 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:50.051085 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010b565-52d8-48e9-b386-16d700fc08ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:50.051970 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010b565-52d8-48e9-b386-16d700fc08ad service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:50.053552 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010b565-52d8-48e9-b386-16d700fc08ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:50.058022 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010b565-52d8-48e9-b386-16d700fc08ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:50.059029 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 823/1634] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 14 19:07:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:07:50.075558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010b565-52d8-48e9-b386-16d700fc08ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:50.081407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010b565-52d8-48e9-b386-16d700fc08ad service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:50.094631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 812/1635] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:07:50 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:07:50.405896 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e15ec9-464c-4d83-b0f9-0d349135ef76 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:50.418269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e15ec9-464c-4d83-b0f9-0d349135ef76 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:50.423867 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273955e3-eeaa-4c46-883c-3ce4afb5ab94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:50.425989 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273955e3-eeaa-4c46-883c-3ce4afb5ab94 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:50.427356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273955e3-eeaa-4c46-883c-3ce4afb5ab94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:50.428270 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273955e3-eeaa-4c46-883c-3ce4afb5ab94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:50.441862 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273955e3-eeaa-4c46-883c-3ce4afb5ab94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:50.443885 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273955e3-eeaa-4c46-883c-3ce4afb5ab94 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:50.459134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 824/1636] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:07:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:07:51.190797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010b565-52d8-48e9-b386-16d700fc08ad service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:51.305386 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010b565-52d8-48e9-b386-16d700fc08ad service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:51.336075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3cf3c8-5f6b-472e-a15f-dbed67ba4d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:51.336075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3cf3c8-5f6b-472e-a15f-dbed67ba4d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:51.336075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3cf3c8-5f6b-472e-a15f-dbed67ba4d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:51.336075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3cf3c8-5f6b-472e-a15f-dbed67ba4d14 admin admin] PATH_INFO: `/v3/users/0cefc934688f4f618c86b882f7447297` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:51.338221 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3cf3c8-5f6b-472e-a15f-dbed67ba4d14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cefc934688f4f618c86b882f7447297)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:51.338221 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: WARNING py.warnings [None req-de3cf3c8-5f6b-472e-a15f-dbed67ba4d14 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 19:07:51.338221 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: warnings.warn(msg) Jul 14 19:07:51.338221 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: Jul 14 19:07:51.339339 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3cf3c8-5f6b-472e-a15f-dbed67ba4d14 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:51.663597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:51.667620 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-de3cf3c8-5f6b-472e-a15f-dbed67ba4d14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cefc934688f4f618c86b882f7447297'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:51.714848 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-de3cf3c8-5f6b-472e-a15f-dbed67ba4d14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cefc934688f4f618c86b882f7447297'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:51.716441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-de3cf3c8-5f6b-472e-a15f-dbed67ba4d14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cefc934688f4f618c86b882f7447297'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:51.722805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-de3cf3c8-5f6b-472e-a15f-dbed67ba4d14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cefc934688f4f618c86b882f7447297'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:51.770701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-de3cf3c8-5f6b-472e-a15f-dbed67ba4d14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cefc934688f4f618c86b882f7447297'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:51.863666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 813/1637] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:07:51 2022] DELETE /identity/v3/users/0cefc934688f4f618c86b882f7447297 => generated 0 bytes in 816 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:07:51.865740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b50266-377c-45a9-ac73-f5ef5d7c89ab None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:51.866227 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b50266-377c-45a9-ac73-f5ef5d7c89ab None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:51.866307 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b50266-377c-45a9-ac73-f5ef5d7c89ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:51.923317 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-68b50266-377c-45a9-ac73-f5ef5d7c89ab None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:51.928719 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-68b50266-377c-45a9-ac73-f5ef5d7c89ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:52.094442 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 825/1638] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:07:52.110270 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3cf3c8-5f6b-472e-a15f-dbed67ba4d14 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:52.113206 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-de3cf3c8-5f6b-472e-a15f-dbed67ba4d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:52.217999 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b50266-377c-45a9-ac73-f5ef5d7c89ab None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:52.222791 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-68b50266-377c-45a9-ac73-f5ef5d7c89ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:52.267016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b50266-377c-45a9-ac73-f5ef5d7c89ab None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:52.278882 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3cf3c8-5f6b-472e-a15f-dbed67ba4d14 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:52.284564 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546700bd-fd8d-48ba-989e-29510d0eafa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:52.286040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546700bd-fd8d-48ba-989e-29510d0eafa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:52.286507 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546700bd-fd8d-48ba-989e-29510d0eafa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:52.286936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546700bd-fd8d-48ba-989e-29510d0eafa0 admin admin] PATH_INFO: `/v3/users/d74fa52ecabd4fa0b0a31f0e2de5d167` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:52.287045 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d432be-aa3f-4b95-a768-9f211759fd6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:52.289528 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d432be-aa3f-4b95-a768-9f211759fd6e service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:52.289696 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546700bd-fd8d-48ba-989e-29510d0eafa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d74fa52ecabd4fa0b0a31f0e2de5d167)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:52.290095 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d432be-aa3f-4b95-a768-9f211759fd6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:52.290564 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d432be-aa3f-4b95-a768-9f211759fd6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:52.294264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546700bd-fd8d-48ba-989e-29510d0eafa0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:52.324594 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d432be-aa3f-4b95-a768-9f211759fd6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:52.332003 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d432be-aa3f-4b95-a768-9f211759fd6e service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:52.351995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 814/1639] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:07:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:07:52.500561 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:52.516718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-546700bd-fd8d-48ba-989e-29510d0eafa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd74fa52ecabd4fa0b0a31f0e2de5d167'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:52.517538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-546700bd-fd8d-48ba-989e-29510d0eafa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd74fa52ecabd4fa0b0a31f0e2de5d167'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:52.561170 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-546700bd-fd8d-48ba-989e-29510d0eafa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd74fa52ecabd4fa0b0a31f0e2de5d167'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:52.562188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-546700bd-fd8d-48ba-989e-29510d0eafa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd74fa52ecabd4fa0b0a31f0e2de5d167'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:52.574374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-546700bd-fd8d-48ba-989e-29510d0eafa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd74fa52ecabd4fa0b0a31f0e2de5d167'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:52.745503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 826/1640] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:07:52 2022] DELETE /identity/v3/users/d74fa52ecabd4fa0b0a31f0e2de5d167 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:07:52.755298 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2887c-ab2f-49a4-a972-d01faec8abb7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:52.756154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2887c-ab2f-49a4-a972-d01faec8abb7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:52.756154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2887c-ab2f-49a4-a972-d01faec8abb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:52.807028 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0ee2887c-ab2f-49a4-a972-d01faec8abb7 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:52.808866 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0ee2887c-ab2f-49a4-a972-d01faec8abb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:52.934501 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 815/1641] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:07:52.994942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546700bd-fd8d-48ba-989e-29510d0eafa0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:52.998368 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-546700bd-fd8d-48ba-989e-29510d0eafa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:53.149784 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546700bd-fd8d-48ba-989e-29510d0eafa0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:53.168305 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2c76a-0410-4d18-b5cd-b39dc92f1140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:53.170490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2c76a-0410-4d18-b5cd-b39dc92f1140 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:53.171101 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2c76a-0410-4d18-b5cd-b39dc92f1140 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:53.171682 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2c76a-0410-4d18-b5cd-b39dc92f1140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:53.203980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2c76a-0410-4d18-b5cd-b39dc92f1140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:53.206015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2c76a-0410-4d18-b5cd-b39dc92f1140 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:53.238754 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 827/1642] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:07:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:07:54.693924 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2887c-ab2f-49a4-a972-d01faec8abb7 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:54.695481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0ee2887c-ab2f-49a4-a972-d01faec8abb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:54.715215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2887c-ab2f-49a4-a972-d01faec8abb7 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:54.748231 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82549d0e-6137-43d2-bc1f-e54eb444e412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:54.750358 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82549d0e-6137-43d2-bc1f-e54eb444e412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:54.750831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82549d0e-6137-43d2-bc1f-e54eb444e412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:54.751479 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82549d0e-6137-43d2-bc1f-e54eb444e412 admin admin] PATH_INFO: `/v3/projects/721bf97931464b0f99e6379267379d32` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:54.754093 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82549d0e-6137-43d2-bc1f-e54eb444e412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=721bf97931464b0f99e6379267379d32)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:54.764087 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82549d0e-6137-43d2-bc1f-e54eb444e412 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:55.033309 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-82549d0e-6137-43d2-bc1f-e54eb444e412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '721bf97931464b0f99e6379267379d32', 'request_id': 'req-82549d0e-6137-43d2-bc1f-e54eb444e412'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:55.062842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2c76a-0410-4d18-b5cd-b39dc92f1140 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:55.065373 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-86f2c76a-0410-4d18-b5cd-b39dc92f1140 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:55.091964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2c76a-0410-4d18-b5cd-b39dc92f1140 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:55.130924 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-82549d0e-6137-43d2-bc1f-e54eb444e412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '721bf97931464b0f99e6379267379d32', 'request_id': 'req-82549d0e-6137-43d2-bc1f-e54eb444e412'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:55.133812 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:55.139616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-82549d0e-6137-43d2-bc1f-e54eb444e412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '721bf97931464b0f99e6379267379d32', 'request_id': 'req-82549d0e-6137-43d2-bc1f-e54eb444e412'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:55.183562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-82549d0e-6137-43d2-bc1f-e54eb444e412 admin admin] The token cache is being invalidate because project 721bf97931464b0f99e6379267379d32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:07:55.183922 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-82549d0e-6137-43d2-bc1f-e54eb444e412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:55.184661 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-82549d0e-6137-43d2-bc1f-e54eb444e412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:55.194580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 816/1643] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:07:54 2022] DELETE /identity/v3/projects/721bf97931464b0f99e6379267379d32 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:07:55.196170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41cc36-a70a-4d4a-acc2-7b3b9102a054 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:55.196467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41cc36-a70a-4d4a-acc2-7b3b9102a054 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:55.197009 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41cc36-a70a-4d4a-acc2-7b3b9102a054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:55.251487 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7b41cc36-a70a-4d4a-acc2-7b3b9102a054 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:55.257480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7b41cc36-a70a-4d4a-acc2-7b3b9102a054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:55.290770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003d2be-ee8d-4212-b7b1-8a39740e6915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:55.293090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003d2be-ee8d-4212-b7b1-8a39740e6915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:55.299064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003d2be-ee8d-4212-b7b1-8a39740e6915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:55.299064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003d2be-ee8d-4212-b7b1-8a39740e6915 admin admin] PATH_INFO: `/v3/projects/8b3c6f140ddd467484b2d73f4ba2fc41` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:55.299064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003d2be-ee8d-4212-b7b1-8a39740e6915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b3c6f140ddd467484b2d73f4ba2fc41)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:55.302134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: WARNING py.warnings [None req-4003d2be-ee8d-4212-b7b1-8a39740e6915 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 19:07:55.302134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: warnings.warn(msg) Jul 14 19:07:55.302134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: Jul 14 19:07:55.307744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003d2be-ee8d-4212-b7b1-8a39740e6915 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:55.443442 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 817/1644] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:55.454540 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a9dd8-0985-4792-b726-3d14f0e7dfe0 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:55.454837 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a9dd8-0985-4792-b726-3d14f0e7dfe0 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:55.455123 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a9dd8-0985-4792-b726-3d14f0e7dfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:55.517380 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-691a9dd8-0985-4792-b726-3d14f0e7dfe0 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:55.522124 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-691a9dd8-0985-4792-b726-3d14f0e7dfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:55.565213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4003d2be-ee8d-4212-b7b1-8a39740e6915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b3c6f140ddd467484b2d73f4ba2fc41', 'request_id': 'req-4003d2be-ee8d-4212-b7b1-8a39740e6915'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:55.680070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 818/1645] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:55.689355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4003d2be-ee8d-4212-b7b1-8a39740e6915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b3c6f140ddd467484b2d73f4ba2fc41', 'request_id': 'req-4003d2be-ee8d-4212-b7b1-8a39740e6915'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:55.696617 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:55.700213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4003d2be-ee8d-4212-b7b1-8a39740e6915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b3c6f140ddd467484b2d73f4ba2fc41', 'request_id': 'req-4003d2be-ee8d-4212-b7b1-8a39740e6915'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:55.703290 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a9dd8-0985-4792-b726-3d14f0e7dfe0 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:55.753260 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a9dd8-0985-4792-b726-3d14f0e7dfe0 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:55.764713 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de39c2-d40f-4cf3-817c-1b04b8a687ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:55.772364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de39c2-d40f-4cf3-817c-1b04b8a687ab admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:55.772364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de39c2-d40f-4cf3-817c-1b04b8a687ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:55.772364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de39c2-d40f-4cf3-817c-1b04b8a687ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:55.772364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de39c2-d40f-4cf3-817c-1b04b8a687ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:07:55.772364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de39c2-d40f-4cf3-817c-1b04b8a687ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:55.776122 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de39c2-d40f-4cf3-817c-1b04b8a687ab admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:55.777992 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4003d2be-ee8d-4212-b7b1-8a39740e6915 admin admin] The token cache is being invalidate because project 8b3c6f140ddd467484b2d73f4ba2fc41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:07:55.778308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4003d2be-ee8d-4212-b7b1-8a39740e6915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:55.779314 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4003d2be-ee8d-4212-b7b1-8a39740e6915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:55.783375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 828/1646] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:07:55 2022] DELETE /identity/v3/projects/8b3c6f140ddd467484b2d73f4ba2fc41 => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:07:55.789231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003d2be-ee8d-4212-b7b1-8a39740e6915 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:55.793738 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4003d2be-ee8d-4212-b7b1-8a39740e6915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:55.827561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 819/1647] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:07:55.832604 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de39c2-d40f-4cf3-817c-1b04b8a687ab admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:55.837019 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-30de39c2-d40f-4cf3-817c-1b04b8a687ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:55.847958 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003d2be-ee8d-4212-b7b1-8a39740e6915 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:55.862201 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5d453-8b97-448a-ae03-eb83c6eb9776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:55.863312 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5d453-8b97-448a-ae03-eb83c6eb9776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:55.865478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5d453-8b97-448a-ae03-eb83c6eb9776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:55.866537 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5d453-8b97-448a-ae03-eb83c6eb9776 admin admin] PATH_INFO: `/v3/users/288c7fdaaba242b68a1b4b649837f5c4` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:55.872578 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5d453-8b97-448a-ae03-eb83c6eb9776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=288c7fdaaba242b68a1b4b649837f5c4)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:55.879770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5d453-8b97-448a-ae03-eb83c6eb9776 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:55.883218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de39c2-d40f-4cf3-817c-1b04b8a687ab admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:55.902121 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fb11a-e168-4123-90ad-5c22bf00356f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:55.903128 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fb11a-e168-4123-90ad-5c22bf00356f admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:55.903448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fb11a-e168-4123-90ad-5c22bf00356f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:55.903729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fb11a-e168-4123-90ad-5c22bf00356f admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:55.908767 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fb11a-e168-4123-90ad-5c22bf00356f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:55.914317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fb11a-e168-4123-90ad-5c22bf00356f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:56.091721 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 820/1648] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:07:55 2022] POST /identity/v3/projects => generated 403 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:56.096495 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fb11a-e168-4123-90ad-5c22bf00356f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:56.164223 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2dd5d453-8b97-448a-ae03-eb83c6eb9776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '288c7fdaaba242b68a1b4b649837f5c4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:56.166907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2dd5d453-8b97-448a-ae03-eb83c6eb9776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '288c7fdaaba242b68a1b4b649837f5c4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:56.213829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2dd5d453-8b97-448a-ae03-eb83c6eb9776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '288c7fdaaba242b68a1b4b649837f5c4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:56.218015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2dd5d453-8b97-448a-ae03-eb83c6eb9776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '288c7fdaaba242b68a1b4b649837f5c4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:56.249158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2dd5d453-8b97-448a-ae03-eb83c6eb9776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '288c7fdaaba242b68a1b4b649837f5c4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:56.285929 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:56.309071 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fb11a-e168-4123-90ad-5c22bf00356f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:56.324298 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3d94f-f4a7-4a64-84ca-11a837a951e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:56.325232 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3d94f-f4a7-4a64-84ca-11a837a951e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:56.327678 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3d94f-f4a7-4a64-84ca-11a837a951e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:56.328584 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3d94f-f4a7-4a64-84ca-11a837a951e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:56.329399 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b3d94f-f4a7-4a64-84ca-11a837a951e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:56.333342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b3d94f-f4a7-4a64-84ca-11a837a951e7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:56.347877 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 829/1649] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:07:55 2022] DELETE /identity/v3/users/288c7fdaaba242b68a1b4b649837f5c4 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:07:56.509196 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 821/1650] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:07:56 2022] POST /identity/v3/users => generated 302 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:07:56.512685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5d453-8b97-448a-ae03-eb83c6eb9776 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:56.519202 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2dd5d453-8b97-448a-ae03-eb83c6eb9776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:56.783007 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5d453-8b97-448a-ae03-eb83c6eb9776 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:56.793608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b65f86-67e4-4c9e-a790-ca25ab0ed290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:56.796972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b65f86-67e4-4c9e-a790-ca25ab0ed290 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:56.797304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b65f86-67e4-4c9e-a790-ca25ab0ed290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:56.797465 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b65f86-67e4-4c9e-a790-ca25ab0ed290 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:56.798452 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b65f86-67e4-4c9e-a790-ca25ab0ed290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:56.800148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b65f86-67e4-4c9e-a790-ca25ab0ed290 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:56.873113 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 830/1651] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:56.875324 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3d94f-f4a7-4a64-84ca-11a837a951e7 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:56.899883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3d94f-f4a7-4a64-84ca-11a837a951e7 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:56.923156 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bc1ed-a934-4e1a-a04a-43d49c7e21cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:56.923156 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bc1ed-a934-4e1a-a04a-43d49c7e21cb admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:56.923156 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bc1ed-a934-4e1a-a04a-43d49c7e21cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:56.923156 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bc1ed-a934-4e1a-a04a-43d49c7e21cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:56.924303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bc1ed-a934-4e1a-a04a-43d49c7e21cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:56.932364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bc1ed-a934-4e1a-a04a-43d49c7e21cb admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:56.951951 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 822/1652] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:56.954078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b65f86-67e4-4c9e-a790-ca25ab0ed290 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:56.973793 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b65f86-67e4-4c9e-a790-ca25ab0ed290 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:56.990408 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a0d4a1-ec0f-432e-b683-84472dc29e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:56.991540 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0d4a1-ec0f-432e-b683-84472dc29e1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:56.991923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0d4a1-ec0f-432e-b683-84472dc29e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:56.992300 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0d4a1-ec0f-432e-b683-84472dc29e1e admin admin] PATH_INFO: `/v3/projects/a51bd679013b4413ae4a34d2436ca3b6/users/678d6766e2b542e58d32400cfafff51d/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:57.041619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0d4a1-ec0f-432e-b683-84472dc29e1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a51bd679013b4413ae4a34d2436ca3b6, user_id=678d6766e2b542e58d32400cfafff51d, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:57.045099 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0d4a1-ec0f-432e-b683-84472dc29e1e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:57.399273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 831/1653] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:07:56 2022] PUT /identity/v3/projects/a51bd679013b4413ae4a34d2436ca3b6/users/678d6766e2b542e58d32400cfafff51d/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:07:57.418157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af095ce-766b-4883-ad05-f59f39c41127 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:57.418841 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af095ce-766b-4883-ad05-f59f39c41127 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:57.419302 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af095ce-766b-4883-ad05-f59f39c41127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:57.481500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0af095ce-766b-4883-ad05-f59f39c41127 None None] MFA Rules not processed for user `678d6766e2b542e58d32400cfafff51d`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:57.482806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0af095ce-766b-4883-ad05-f59f39c41127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:57.653047 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 823/1654] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:57.658134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a0d4a1-ec0f-432e-b683-84472dc29e1e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:57.734335 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:57.808689 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a0d4a1-ec0f-432e-b683-84472dc29e1e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:57.832200 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3c34a-6f31-4756-9426-7d01b7269117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:57.832923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3c34a-6f31-4756-9426-7d01b7269117 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:57.833176 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3c34a-6f31-4756-9426-7d01b7269117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:57.833453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3c34a-6f31-4756-9426-7d01b7269117 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:57.841903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3c34a-6f31-4756-9426-7d01b7269117 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:57.847654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3c34a-6f31-4756-9426-7d01b7269117 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:57.967825 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 832/1655] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:07:57 2022] POST /identity/v3/projects => generated 405 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:57.972078 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af095ce-766b-4883-ad05-f59f39c41127 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:57.989478 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af095ce-766b-4883-ad05-f59f39c41127 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:58.168894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd03be-147d-4574-87a3-09cf24ab7fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:58.170179 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd03be-147d-4574-87a3-09cf24ab7fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:58.170467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd03be-147d-4574-87a3-09cf24ab7fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:58.170745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd03be-147d-4574-87a3-09cf24ab7fa7 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:58.171484 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd03be-147d-4574-87a3-09cf24ab7fa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:58.173796 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd03be-147d-4574-87a3-09cf24ab7fa7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:58.301628 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 824/1656] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:07:57 2022] POST /identity/v3/users => generated 309 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:58.303565 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3c34a-6f31-4756-9426-7d01b7269117 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:58.318407 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3c34a-6f31-4756-9426-7d01b7269117 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:58.330780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0f3da-e24a-4a23-a618-b0697599808f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:58.333606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0f3da-e24a-4a23-a618-b0697599808f admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:58.333983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0f3da-e24a-4a23-a618-b0697599808f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:58.335235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0f3da-e24a-4a23-a618-b0697599808f admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:58.335759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0f3da-e24a-4a23-a618-b0697599808f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:58.339749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0f3da-e24a-4a23-a618-b0697599808f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:58.378113 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 833/1657] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:07:58.381342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd03be-147d-4574-87a3-09cf24ab7fa7 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:58.400318 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd03be-147d-4574-87a3-09cf24ab7fa7 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:58.426501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0f3da-e24a-4a23-a618-b0697599808f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:58.426501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4bb0f3da-e24a-4a23-a618-b0697599808f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:58.427099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d8450-4855-4532-8002-d49761e9a98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:58.427099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d8450-4855-4532-8002-d49761e9a98f admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:58.427099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d8450-4855-4532-8002-d49761e9a98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:58.427099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d8450-4855-4532-8002-d49761e9a98f admin admin] PATH_INFO: `/v3/projects/249678c9fcd64c478017877a077720ee/users/4c50d1ecabcf4475b7aed8e0613a4cee/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:58.440025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0f3da-e24a-4a23-a618-b0697599808f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:58.459818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b462727-8aa5-4ae4-809b-45bdd177a3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:58.464837 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b462727-8aa5-4ae4-809b-45bdd177a3c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:58.465591 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b462727-8aa5-4ae4-809b-45bdd177a3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:58.466128 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b462727-8aa5-4ae4-809b-45bdd177a3c1 admin admin] PATH_INFO: `/v3/projects/3b1a80c28e164a6c834d0043ccf5d2d0` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:58.473425 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d8450-4855-4532-8002-d49761e9a98f admin admin] RBAC: Authorizing `identity:create_grant(project_id=249678c9fcd64c478017877a077720ee, user_id=4c50d1ecabcf4475b7aed8e0613a4cee, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:58.473635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b462727-8aa5-4ae4-809b-45bdd177a3c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b1a80c28e164a6c834d0043ccf5d2d0)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:58.478550 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b462727-8aa5-4ae4-809b-45bdd177a3c1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:58.482192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d8450-4855-4532-8002-d49761e9a98f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:58.589501 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 825/1658] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:07:58 2022] PUT /identity/v3/projects/249678c9fcd64c478017877a077720ee/users/4c50d1ecabcf4475b7aed8e0613a4cee/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:07:58.595439 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d8450-4855-4532-8002-d49761e9a98f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:58.641533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4b462727-8aa5-4ae4-809b-45bdd177a3c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b1a80c28e164a6c834d0043ccf5d2d0', 'request_id': 'req-4b462727-8aa5-4ae4-809b-45bdd177a3c1'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:58.654731 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:58.731900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d8450-4855-4532-8002-d49761e9a98f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:58.746053 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4b462727-8aa5-4ae4-809b-45bdd177a3c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b1a80c28e164a6c834d0043ccf5d2d0', 'request_id': 'req-4b462727-8aa5-4ae4-809b-45bdd177a3c1'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:58.756103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4b462727-8aa5-4ae4-809b-45bdd177a3c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b1a80c28e164a6c834d0043ccf5d2d0', 'request_id': 'req-4b462727-8aa5-4ae4-809b-45bdd177a3c1'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:58.779519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4b462727-8aa5-4ae4-809b-45bdd177a3c1 admin admin] The token cache is being invalidate because project 3b1a80c28e164a6c834d0043ccf5d2d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:07:58.792633 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4b462727-8aa5-4ae4-809b-45bdd177a3c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:58.798373 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4b462727-8aa5-4ae4-809b-45bdd177a3c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:07:58.814996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 834/1659] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:07:58 2022] DELETE /identity/v3/projects/3b1a80c28e164a6c834d0043ccf5d2d0 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:07:58.817020 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6263af-91de-44f6-b211-5c34af321b1c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:58.819419 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6263af-91de-44f6-b211-5c34af321b1c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:58.819996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6263af-91de-44f6-b211-5c34af321b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:58.835777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942f011-ee0b-4959-968c-4a655ef49ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:58.836840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942f011-ee0b-4959-968c-4a655ef49ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:58.837275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942f011-ee0b-4959-968c-4a655ef49ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:58.837812 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942f011-ee0b-4959-968c-4a655ef49ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:58.838901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942f011-ee0b-4959-968c-4a655ef49ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:58.840986 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942f011-ee0b-4959-968c-4a655ef49ce2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:58.864510 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 826/1660] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:07:58.868953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942f011-ee0b-4959-968c-4a655ef49ce2 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:58.873749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0942f011-ee0b-4959-968c-4a655ef49ce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:58.902120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9d6263af-91de-44f6-b211-5c34af321b1c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:58.903778 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9d6263af-91de-44f6-b211-5c34af321b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:58.914159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942f011-ee0b-4959-968c-4a655ef49ce2 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:58.920777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67b2a5-8be1-43a3-bb86-ec4a0f6f8e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:58.921886 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67b2a5-8be1-43a3-bb86-ec4a0f6f8e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:58.922181 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67b2a5-8be1-43a3-bb86-ec4a0f6f8e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:58.922467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67b2a5-8be1-43a3-bb86-ec4a0f6f8e90 admin admin] PATH_INFO: `/v3/projects/249678c9fcd64c478017877a077720ee/users/4c50d1ecabcf4475b7aed8e0613a4cee/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:58.924650 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67b2a5-8be1-43a3-bb86-ec4a0f6f8e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=249678c9fcd64c478017877a077720ee, user_id=4c50d1ecabcf4475b7aed8e0613a4cee, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:58.928643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67b2a5-8be1-43a3-bb86-ec4a0f6f8e90 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:58.937442 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 835/1661] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:58.946211 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504f27b-3016-477b-a48f-679c64b4cf2f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:58.963219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504f27b-3016-477b-a48f-679c64b4cf2f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:58.963458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 827/1662] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:07:58 2022] PUT /identity/v3/projects/249678c9fcd64c478017877a077720ee/users/4c50d1ecabcf4475b7aed8e0613a4cee/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:07:58.963629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504f27b-3016-477b-a48f-679c64b4cf2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:58.995450 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5504f27b-3016-477b-a48f-679c64b4cf2f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:58.997280 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5504f27b-3016-477b-a48f-679c64b4cf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:58.998696 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c5566-4235-462e-8f3a-8d5bf062f088 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:58.999723 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c5566-4235-462e-8f3a-8d5bf062f088 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:59.000345 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c5566-4235-462e-8f3a-8d5bf062f088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:59.110964 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-939c5566-4235-462e-8f3a-8d5bf062f088 None None] MFA Rules not processed for user `4c50d1ecabcf4475b7aed8e0613a4cee`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:07:59.112934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-939c5566-4235-462e-8f3a-8d5bf062f088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:59.216869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 836/1663] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:59.216869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504f27b-3016-477b-a48f-679c64b4cf2f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:59.256992 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504f27b-3016-477b-a48f-679c64b4cf2f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:59.293443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013155ab-cdc7-48e0-b3b7-53c6b38b64e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:59.312265 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013155ab-cdc7-48e0-b3b7-53c6b38b64e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:59.312670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013155ab-cdc7-48e0-b3b7-53c6b38b64e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:59.313038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013155ab-cdc7-48e0-b3b7-53c6b38b64e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:59.316278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013155ab-cdc7-48e0-b3b7-53c6b38b64e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:07:59.316666 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013155ab-cdc7-48e0-b3b7-53c6b38b64e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:59.326737 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013155ab-cdc7-48e0-b3b7-53c6b38b64e0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:59.395875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 837/1664] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:07:59.401035 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013155ab-cdc7-48e0-b3b7-53c6b38b64e0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:59.444042 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013155ab-cdc7-48e0-b3b7-53c6b38b64e0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:59.457462 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 828/1665] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:07:58 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:07:59.471833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c5566-4235-462e-8f3a-8d5bf062f088 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:59.475583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-939c5566-4235-462e-8f3a-8d5bf062f088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:07:59.487422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe25904-d72d-4d57-8df0-1bfc834f7d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:59.492223 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe25904-d72d-4d57-8df0-1bfc834f7d79 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:59.493437 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe25904-d72d-4d57-8df0-1bfc834f7d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:59.494428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe25904-d72d-4d57-8df0-1bfc834f7d79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:59.498660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe25904-d72d-4d57-8df0-1bfc834f7d79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:59.508848 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe25904-d72d-4d57-8df0-1bfc834f7d79 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:07:59.534044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:59.622736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:07:59.652902 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 838/1666] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:07:59 2022] POST /identity/v3/projects => generated 365 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:07:59.659213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe25904-d72d-4d57-8df0-1bfc834f7d79 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:07:59.665019 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c5566-4235-462e-8f3a-8d5bf062f088 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:59.803679 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab8e07-9302-4305-863d-f7253177a2f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:59.804599 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab8e07-9302-4305-863d-f7253177a2f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:59.804964 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab8e07-9302-4305-863d-f7253177a2f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:59.805314 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab8e07-9302-4305-863d-f7253177a2f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:59.909603 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe25904-d72d-4d57-8df0-1bfc834f7d79 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:07:59.923114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099c98a-d89a-4949-a726-552fc354d1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:07:59.924695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099c98a-d89a-4949-a726-552fc354d1f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:07:59.925402 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099c98a-d89a-4949-a726-552fc354d1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:07:59.926055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099c98a-d89a-4949-a726-552fc354d1f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:07:59.927646 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099c98a-d89a-4949-a726-552fc354d1f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:07:59.932932 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099c98a-d89a-4949-a726-552fc354d1f3 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:00.021162 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab8e07-9302-4305-863d-f7253177a2f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:00.027277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab8e07-9302-4305-863d-f7253177a2f7 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:00.046847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 839/1667] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:07:59 2022] POST /identity/v3/users => generated 283 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:00.049741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67dba3-498c-48d3-bd4f-6ac60fa114e1 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:00.049839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67dba3-498c-48d3-bd4f-6ac60fa114e1 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:00.050081 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67dba3-498c-48d3-bd4f-6ac60fa114e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:00.076905 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 829/1668] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:07:59 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:08:00.088173 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab8e07-9302-4305-863d-f7253177a2f7 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:00.093555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6a67dba3-498c-48d3-bd4f-6ac60fa114e1 None None] MFA Rules not processed for user `4a5c22d1844642418e4478006c5fb1ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:00.095100 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6a67dba3-498c-48d3-bd4f-6ac60fa114e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:00.152515 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab8e07-9302-4305-863d-f7253177a2f7 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:00.170060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b688521-b726-43c1-bf82-041ba766ac3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:00.170931 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b688521-b726-43c1-bf82-041ba766ac3d admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:00.171343 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b688521-b726-43c1-bf82-041ba766ac3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:00.171762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b688521-b726-43c1-bf82-041ba766ac3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:00.172497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b688521-b726-43c1-bf82-041ba766ac3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:00.174212 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b688521-b726-43c1-bf82-041ba766ac3d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:00.193291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 840/1669] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:00.205263 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67dba3-498c-48d3-bd4f-6ac60fa114e1 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:00.212344 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 830/1670] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:00.213994 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6005a5e-4de1-4ba3-aaed-5ce02d57a097 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:00.214294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6005a5e-4de1-4ba3-aaed-5ce02d57a097 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:00.214653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6005a5e-4de1-4ba3-aaed-5ce02d57a097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:00.249316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67dba3-498c-48d3-bd4f-6ac60fa114e1 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:00.262471 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c92de3-dfd8-473a-97f6-4576087c9af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:00.263981 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c92de3-dfd8-473a-97f6-4576087c9af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:00.264727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c92de3-dfd8-473a-97f6-4576087c9af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:00.266735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c92de3-dfd8-473a-97f6-4576087c9af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:00.267700 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c92de3-dfd8-473a-97f6-4576087c9af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:00.270363 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b6005a5e-4de1-4ba3-aaed-5ce02d57a097 None None] MFA Rules not processed for user `4a5c22d1844642418e4478006c5fb1ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:00.271757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c92de3-dfd8-473a-97f6-4576087c9af3 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:00.279603 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b6005a5e-4de1-4ba3-aaed-5ce02d57a097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:00.288754 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 831/1671] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:00.302459 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6005a5e-4de1-4ba3-aaed-5ce02d57a097 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:00.307717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 841/1672] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:00.309690 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b6005a5e-4de1-4ba3-aaed-5ce02d57a097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:00.313958 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c92de3-dfd8-473a-97f6-4576087c9af3 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:00.335068 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c92de3-dfd8-473a-97f6-4576087c9af3 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:00.348383 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584c5c2-450c-41d5-b70f-626cf8c42a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:00.349756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584c5c2-450c-41d5-b70f-626cf8c42a1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:00.350047 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584c5c2-450c-41d5-b70f-626cf8c42a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:00.350876 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584c5c2-450c-41d5-b70f-626cf8c42a1d admin admin] PATH_INFO: `/v3/projects/65d1be152dc44aafb1178d275e152aec/users/e0c759aaf5244c7fb85d2df571d9e349/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:00.386403 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584c5c2-450c-41d5-b70f-626cf8c42a1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=65d1be152dc44aafb1178d275e152aec, user_id=e0c759aaf5244c7fb85d2df571d9e349, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:00.392911 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584c5c2-450c-41d5-b70f-626cf8c42a1d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:00.459198 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6005a5e-4de1-4ba3-aaed-5ce02d57a097 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:00.469683 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abb524-3898-430e-a009-b668194b3a24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:00.470822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abb524-3898-430e-a009-b668194b3a24 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:00.471419 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abb524-3898-430e-a009-b668194b3a24 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:00.473022 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abb524-3898-430e-a009-b668194b3a24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:00.492525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abb524-3898-430e-a009-b668194b3a24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:00.492525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abb524-3898-430e-a009-b668194b3a24 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:00.512171 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 832/1673] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:00.514391 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abb524-3898-430e-a009-b668194b3a24 service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:00.525481 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 842/1674] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:00 2022] PUT /identity/v3/projects/65d1be152dc44aafb1178d275e152aec/users/e0c759aaf5244c7fb85d2df571d9e349/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:00.538663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81283822-4027-4e3c-9a8a-ddf51c27990e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:00.544098 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81283822-4027-4e3c-9a8a-ddf51c27990e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:00.544098 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81283822-4027-4e3c-9a8a-ddf51c27990e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:00.544357 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abb524-3898-430e-a009-b668194b3a24 service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:00.581623 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:00.633184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-81283822-4027-4e3c-9a8a-ddf51c27990e None None] MFA Rules not processed for user `e0c759aaf5244c7fb85d2df571d9e349`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:00.634541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-81283822-4027-4e3c-9a8a-ddf51c27990e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:00.710176 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:00.742884 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ce4c7-dc08-4639-b046-da07e305b9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:00.743956 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce4c7-dc08-4639-b046-da07e305b9e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:00.744343 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce4c7-dc08-4639-b046-da07e305b9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:00.744740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce4c7-dc08-4639-b046-da07e305b9e5 admin admin] PATH_INFO: `/v3/users/678d6766e2b542e58d32400cfafff51d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:00.747061 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ce4c7-dc08-4639-b046-da07e305b9e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=678d6766e2b542e58d32400cfafff51d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:00.752114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ce4c7-dc08-4639-b046-da07e305b9e5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:00.844996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 843/1675] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:00 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:00.849374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81283822-4027-4e3c-9a8a-ddf51c27990e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:00.862996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81283822-4027-4e3c-9a8a-ddf51c27990e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:00.869349 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eca870-0b7b-47f3-af25-54908eb96ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:00.871320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eca870-0b7b-47f3-af25-54908eb96ebe admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:00.871320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eca870-0b7b-47f3-af25-54908eb96ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:00.871320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eca870-0b7b-47f3-af25-54908eb96ebe admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:00.871585 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eca870-0b7b-47f3-af25-54908eb96ebe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:00.873602 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eca870-0b7b-47f3-af25-54908eb96ebe admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:01.027061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 844/1676] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:08:00 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:01.027061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eca870-0b7b-47f3-af25-54908eb96ebe admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:01.102773 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-841ce4c7-dc08-4639-b046-da07e305b9e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '678d6766e2b542e58d32400cfafff51d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:01.118263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-841ce4c7-dc08-4639-b046-da07e305b9e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '678d6766e2b542e58d32400cfafff51d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:01.123068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-841ce4c7-dc08-4639-b046-da07e305b9e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '678d6766e2b542e58d32400cfafff51d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:01.123068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-841ce4c7-dc08-4639-b046-da07e305b9e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '678d6766e2b542e58d32400cfafff51d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:01.166631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-841ce4c7-dc08-4639-b046-da07e305b9e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '678d6766e2b542e58d32400cfafff51d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:01.203588 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eca870-0b7b-47f3-af25-54908eb96ebe admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:01.222822 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb05e7-afe4-4842-98bb-4c39c2fa3d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:01.225859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb05e7-afe4-4842-98bb-4c39c2fa3d7e admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:01.225859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb05e7-afe4-4842-98bb-4c39c2fa3d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:01.225859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb05e7-afe4-4842-98bb-4c39c2fa3d7e admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:01.225859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb05e7-afe4-4842-98bb-4c39c2fa3d7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:01.227570 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb05e7-afe4-4842-98bb-4c39c2fa3d7e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:01.256995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 833/1677] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:08:00 2022] DELETE /identity/v3/users/678d6766e2b542e58d32400cfafff51d => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:01.261411 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9326b7-8ff3-40bd-be90-569370c2f31b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:01.261709 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9326b7-8ff3-40bd-be90-569370c2f31b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:01.262068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9326b7-8ff3-40bd-be90-569370c2f31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:01.299793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0d9326b7-8ff3-40bd-be90-569370c2f31b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:01.300746 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0d9326b7-8ff3-40bd-be90-569370c2f31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:01.355130 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 845/1678] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:08:01 2022] POST /identity/v3/users => generated 289 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:01.361973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb05e7-afe4-4842-98bb-4c39c2fa3d7e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:01.369517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-85cb05e7-afe4-4842-98bb-4c39c2fa3d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:01.419932 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb05e7-afe4-4842-98bb-4c39c2fa3d7e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:01.500895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 834/1679] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:01.500895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472480fe-ea87-4957-9691-25551e57c380 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:01.500895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472480fe-ea87-4957-9691-25551e57c380 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:01.500895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472480fe-ea87-4957-9691-25551e57c380 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:01.515569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 835/1680] 10.222.0.69 () {54 vars in 912 bytes} [Thu Jul 14 19:08:01 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:08:01.516562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472480fe-ea87-4957-9691-25551e57c380 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:01.526404 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-472480fe-ea87-4957-9691-25551e57c380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:01.717748 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9f9ed-7378-4826-ad74-34a145fd8e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:01.725153 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9f9ed-7378-4826-ad74-34a145fd8e31 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:01.726919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9f9ed-7378-4826-ad74-34a145fd8e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:01.727132 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9f9ed-7378-4826-ad74-34a145fd8e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:01.736855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f9f9ed-7378-4826-ad74-34a145fd8e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:01.749935 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f9f9ed-7378-4826-ad74-34a145fd8e31 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:01.790686 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:01.803512 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 846/1681] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:01.808598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627ec55-09e3-4cd1-813b-276bd7876b75 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:01.809075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627ec55-09e3-4cd1-813b-276bd7876b75 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:01.809538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627ec55-09e3-4cd1-813b-276bd7876b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:01.850803 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472480fe-ea87-4957-9691-25551e57c380 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:01.864855 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9e441-3f78-4371-ad5e-1a273953b057 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:01.871153 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9e441-3f78-4371-ad5e-1a273953b057 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:01.873154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9e441-3f78-4371-ad5e-1a273953b057 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:01.874549 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9e441-3f78-4371-ad5e-1a273953b057 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:01.894687 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9e441-3f78-4371-ad5e-1a273953b057 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:01.898079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9e441-3f78-4371-ad5e-1a273953b057 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:01.914752 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2627ec55-09e3-4cd1-813b-276bd7876b75 None None] MFA Rules not processed for user `74b8c75b9cce442b8c430fab6a3fea17`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:01.916305 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2627ec55-09e3-4cd1-813b-276bd7876b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:01.934530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 836/1682] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:01.937403 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9e441-3f78-4371-ad5e-1a273953b057 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:01.949824 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 847/1683] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 14 19:08:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:01.956953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9e441-3f78-4371-ad5e-1a273953b057 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:01.967452 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34c8b7-756e-4beb-bee8-15c0c46dacb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:01.968781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34c8b7-756e-4beb-bee8-15c0c46dacb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:01.969222 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34c8b7-756e-4beb-bee8-15c0c46dacb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:01.969609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34c8b7-756e-4beb-bee8-15c0c46dacb4 admin admin] PATH_INFO: `/v3/projects/6daccaf10bc341268a69861e3b23206b/users/491d2a0af36e473d84d5d54e2acba117/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:01.999238 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34c8b7-756e-4beb-bee8-15c0c46dacb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6daccaf10bc341268a69861e3b23206b, user_id=491d2a0af36e473d84d5d54e2acba117, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:02.001531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34c8b7-756e-4beb-bee8-15c0c46dacb4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:02.106638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 837/1684] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:01 2022] PUT /identity/v3/projects/6daccaf10bc341268a69861e3b23206b/users/491d2a0af36e473d84d5d54e2acba117/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:02.112052 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627ec55-09e3-4cd1-813b-276bd7876b75 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:02.140627 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627ec55-09e3-4cd1-813b-276bd7876b75 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:02.144923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:02.289134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288326bc-2f91-49f7-874b-4da41745b4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:02.290370 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288326bc-2f91-49f7-874b-4da41745b4f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:02.295578 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288326bc-2f91-49f7-874b-4da41745b4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:02.296023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288326bc-2f91-49f7-874b-4da41745b4f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:02.298349 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288326bc-2f91-49f7-874b-4da41745b4f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:02.305574 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288326bc-2f91-49f7-874b-4da41745b4f7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:02.338064 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34c8b7-756e-4beb-bee8-15c0c46dacb4 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:02.385344 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 848/1685] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:02.387802 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288326bc-2f91-49f7-874b-4da41745b4f7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:02.449564 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288326bc-2f91-49f7-874b-4da41745b4f7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:02.481690 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4faf224-558d-469b-a53e-9681ecb8b3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:02.493820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4faf224-558d-469b-a53e-9681ecb8b3c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:02.493820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4faf224-558d-469b-a53e-9681ecb8b3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:02.493820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4faf224-558d-469b-a53e-9681ecb8b3c1 admin admin] PATH_INFO: `/v3/projects/6daccaf10bc341268a69861e3b23206b/users/491d2a0af36e473d84d5d54e2acba117/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:02.498141 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4faf224-558d-469b-a53e-9681ecb8b3c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6daccaf10bc341268a69861e3b23206b, user_id=491d2a0af36e473d84d5d54e2acba117, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:02.501120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4faf224-558d-469b-a53e-9681ecb8b3c1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:02.530827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 849/1686] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:02 2022] PUT /identity/v3/projects/6daccaf10bc341268a69861e3b23206b/users/491d2a0af36e473d84d5d54e2acba117/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:08:02.557724 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02fc6a-09fb-407f-bd8c-ab9a37f83db3 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:02.559117 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02fc6a-09fb-407f-bd8c-ab9a37f83db3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:02.559664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02fc6a-09fb-407f-bd8c-ab9a37f83db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:02.585478 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34c8b7-756e-4beb-bee8-15c0c46dacb4 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:02.656719 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7f02fc6a-09fb-407f-bd8c-ab9a37f83db3 None None] MFA Rules not processed for user `491d2a0af36e473d84d5d54e2acba117`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:02.658338 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7f02fc6a-09fb-407f-bd8c-ab9a37f83db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:02.863893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221f63a-8834-4df5-80a7-753e6c7b5646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:02.864766 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221f63a-8834-4df5-80a7-753e6c7b5646 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:02.865197 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221f63a-8834-4df5-80a7-753e6c7b5646 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:02.865614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221f63a-8834-4df5-80a7-753e6c7b5646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:02.885363 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f221f63a-8834-4df5-80a7-753e6c7b5646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:02.887755 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f221f63a-8834-4df5-80a7-753e6c7b5646 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:02.915942 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 838/1687] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:08:02.944561 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 850/1688] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:08:02.959230 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221f63a-8834-4df5-80a7-753e6c7b5646 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:02.962000 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f221f63a-8834-4df5-80a7-753e6c7b5646 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:03.093074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221f63a-8834-4df5-80a7-753e6c7b5646 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:03.100078 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcfffc-cc63-4159-951a-9a4b4f649637 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:03.101041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcfffc-cc63-4159-951a-9a4b4f649637 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:03.101479 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcfffc-cc63-4159-951a-9a4b4f649637 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:03.101956 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcfffc-cc63-4159-951a-9a4b4f649637 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:03.119041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafcfffc-cc63-4159-951a-9a4b4f649637 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:03.126504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafcfffc-cc63-4159-951a-9a4b4f649637 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:03.151987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 839/1689] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:03.504526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02fc6a-09fb-407f-bd8c-ab9a37f83db3 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:03.521566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02fc6a-09fb-407f-bd8c-ab9a37f83db3 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:03.535673 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2f52d-f8cd-49e8-b172-d5f452599eae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:03.537260 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2f52d-f8cd-49e8-b172-d5f452599eae service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:03.538429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2f52d-f8cd-49e8-b172-d5f452599eae service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:03.539277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2f52d-f8cd-49e8-b172-d5f452599eae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:03.547760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-39c2f52d-f8cd-49e8-b172-d5f452599eae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:03.754228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2f52d-f8cd-49e8-b172-d5f452599eae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:03.755797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2f52d-f8cd-49e8-b172-d5f452599eae service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:03.804393 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 851/1690] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:03 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:04.302318 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288c594-8037-447e-9cf4-feda773dcfed None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:04.302875 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288c594-8037-447e-9cf4-feda773dcfed None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:04.302875 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288c594-8037-447e-9cf4-feda773dcfed None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:04.305000 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 840/1691] 10.222.0.69 () {54 vars in 960 bytes} [Thu Jul 14 19:08:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:08:04.311769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db48dd-b392-4d73-ad6b-93554e2c334c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:04.312490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db48dd-b392-4d73-ad6b-93554e2c334c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:04.312490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db48dd-b392-4d73-ad6b-93554e2c334c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:04.362441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e9db48dd-b392-4d73-ad6b-93554e2c334c None None] MFA Rules not processed for user `c577fa8670e34449a80de0c8e0a15a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:04.365443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e9db48dd-b392-4d73-ad6b-93554e2c334c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:04.411701 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:04.619208 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288c594-8037-447e-9cf4-feda773dcfed None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:04.626392 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1288c594-8037-447e-9cf4-feda773dcfed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:04.726005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:04.740084 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 852/1692] 10.222.0.69 () {58 vars in 1057 bytes} [Thu Jul 14 19:08:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:08:04.741326 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db48dd-b392-4d73-ad6b-93554e2c334c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:04.792353 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db48dd-b392-4d73-ad6b-93554e2c334c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:04.807842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee60123-e7d9-4ae2-a57a-259499a75167 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c577fa8670e34449a80de0c8e0a15a8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:04.808529 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee60123-e7d9-4ae2-a57a-259499a75167 service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:04.808820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee60123-e7d9-4ae2-a57a-259499a75167 service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:04.809101 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee60123-e7d9-4ae2-a57a-259499a75167 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:04.819706 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9ee60123-e7d9-4ae2-a57a-259499a75167 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:04.852963 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee60123-e7d9-4ae2-a57a-259499a75167 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:04.856626 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee60123-e7d9-4ae2-a57a-259499a75167 service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:04.924764 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 853/1693] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:04.925773 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee60123-e7d9-4ae2-a57a-259499a75167 service placement] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:04.936079 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9ee60123-e7d9-4ae2-a57a-259499a75167 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:04.939064 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288c594-8037-447e-9cf4-feda773dcfed None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:04.954080 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b278f-124a-4827-bea9-3d27bf26b09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:04.957795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b278f-124a-4827-bea9-3d27bf26b09d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:04.957795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b278f-124a-4827-bea9-3d27bf26b09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:04.957795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b278f-124a-4827-bea9-3d27bf26b09d admin admin] PATH_INFO: `/v3/projects/a51bd679013b4413ae4a34d2436ca3b6` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:04.958928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b278f-124a-4827-bea9-3d27bf26b09d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a51bd679013b4413ae4a34d2436ca3b6)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:04.961879 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b278f-124a-4827-bea9-3d27bf26b09d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:04.999014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee60123-e7d9-4ae2-a57a-259499a75167 service placement] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:05.023930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8bf3e-95b5-49b2-8952-532366c807dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:05.025421 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8bf3e-95b5-49b2-8952-532366c807dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:05.026676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8bf3e-95b5-49b2-8952-532366c807dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:05.027531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8bf3e-95b5-49b2-8952-532366c807dd admin admin] PATH_INFO: `/v3/users/9d1d7a0d45b040f2a927b299638d3c44` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:05.031156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8bf3e-95b5-49b2-8952-532366c807dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d1d7a0d45b040f2a927b299638d3c44)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:05.040048 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8bf3e-95b5-49b2-8952-532366c807dd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:05.182576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-649b278f-124a-4827-bea9-3d27bf26b09d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a51bd679013b4413ae4a34d2436ca3b6', 'request_id': 'req-649b278f-124a-4827-bea9-3d27bf26b09d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:05.342607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-649b278f-124a-4827-bea9-3d27bf26b09d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a51bd679013b4413ae4a34d2436ca3b6', 'request_id': 'req-649b278f-124a-4827-bea9-3d27bf26b09d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:05.352898 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-649b278f-124a-4827-bea9-3d27bf26b09d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a51bd679013b4413ae4a34d2436ca3b6', 'request_id': 'req-649b278f-124a-4827-bea9-3d27bf26b09d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:05.379682 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2da8bf3e-95b5-49b2-8952-532366c807dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d1d7a0d45b040f2a927b299638d3c44'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:05.380861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2da8bf3e-95b5-49b2-8952-532366c807dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d1d7a0d45b040f2a927b299638d3c44'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:05.423100 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-649b278f-124a-4827-bea9-3d27bf26b09d admin admin] The token cache is being invalidate because project a51bd679013b4413ae4a34d2436ca3b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:08:05.423448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-649b278f-124a-4827-bea9-3d27bf26b09d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:05.425210 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-649b278f-124a-4827-bea9-3d27bf26b09d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:05.427968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 841/1694] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:08:04 2022] DELETE /identity/v3/projects/a51bd679013b4413ae4a34d2436ca3b6 => generated 0 bytes in 809 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:05.436085 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b278f-124a-4827-bea9-3d27bf26b09d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:05.445790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-649b278f-124a-4827-bea9-3d27bf26b09d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:05.447080 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2da8bf3e-95b5-49b2-8952-532366c807dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d1d7a0d45b040f2a927b299638d3c44'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:05.448385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2da8bf3e-95b5-49b2-8952-532366c807dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d1d7a0d45b040f2a927b299638d3c44'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:05.457365 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2da8bf3e-95b5-49b2-8952-532366c807dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d1d7a0d45b040f2a927b299638d3c44'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:05.552281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 854/1695] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:08:04 2022] DELETE /identity/v3/users/9d1d7a0d45b040f2a927b299638d3c44 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:05.557230 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bd662-da07-4327-bc02-edbb7235e149 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:05.557837 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bd662-da07-4327-bc02-edbb7235e149 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:05.558290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bd662-da07-4327-bc02-edbb7235e149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:05.640331 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e93bd662-da07-4327-bc02-edbb7235e149 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:05.641987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e93bd662-da07-4327-bc02-edbb7235e149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:05.752103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b278f-124a-4827-bea9-3d27bf26b09d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:05.901226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 855/1696] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:08:05.931431 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bd662-da07-4327-bc02-edbb7235e149 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:05.933491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e93bd662-da07-4327-bc02-edbb7235e149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:05.963117 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:05.968794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:06.007227 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96910291-7b28-445b-9d35-2450cd62d8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:06.008011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96910291-7b28-445b-9d35-2450cd62d8d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:06.008497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96910291-7b28-445b-9d35-2450cd62d8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:06.008938 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96910291-7b28-445b-9d35-2450cd62d8d1 admin admin] PATH_INFO: `/v3/users/4c50d1ecabcf4475b7aed8e0613a4cee` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:06.018841 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96910291-7b28-445b-9d35-2450cd62d8d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c50d1ecabcf4475b7aed8e0613a4cee)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:06.029004 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96910291-7b28-445b-9d35-2450cd62d8d1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:06.177544 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bd662-da07-4327-bc02-edbb7235e149 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:06.191907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416dfc9-ddaf-42d1-966b-49298848303c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:06.192588 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416dfc9-ddaf-42d1-966b-49298848303c service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:06.192933 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416dfc9-ddaf-42d1-966b-49298848303c service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:06.193294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416dfc9-ddaf-42d1-966b-49298848303c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:06.228504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416dfc9-ddaf-42d1-966b-49298848303c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:06.233004 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416dfc9-ddaf-42d1-966b-49298848303c service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:06.260923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 856/1697] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:08:06.358328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-96910291-7b28-445b-9d35-2450cd62d8d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c50d1ecabcf4475b7aed8e0613a4cee'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:06.378013 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-96910291-7b28-445b-9d35-2450cd62d8d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c50d1ecabcf4475b7aed8e0613a4cee'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:06.379804 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-96910291-7b28-445b-9d35-2450cd62d8d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c50d1ecabcf4475b7aed8e0613a4cee'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:06.381437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-96910291-7b28-445b-9d35-2450cd62d8d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c50d1ecabcf4475b7aed8e0613a4cee'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:06.420606 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-96910291-7b28-445b-9d35-2450cd62d8d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c50d1ecabcf4475b7aed8e0613a4cee'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:06.542956 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 842/1698] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:08:05 2022] DELETE /identity/v3/users/4c50d1ecabcf4475b7aed8e0613a4cee => generated 0 bytes in 1109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:06.947066 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a645ae6-b73c-4b78-adde-e19c6149adde None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:06.947603 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a645ae6-b73c-4b78-adde-e19c6149adde None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:06.947695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a645ae6-b73c-4b78-adde-e19c6149adde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:07.023552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3a645ae6-b73c-4b78-adde-e19c6149adde None None] MFA Rules not processed for user `7d5fa180ee554e8b8e46321892969489`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:07.026264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3a645ae6-b73c-4b78-adde-e19c6149adde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:07.192939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 857/1699] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:07.201492 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96910291-7b28-445b-9d35-2450cd62d8d1 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:07.205938 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-96910291-7b28-445b-9d35-2450cd62d8d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:07.376455 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96910291-7b28-445b-9d35-2450cd62d8d1 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:07.393244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d14e6-2b52-45d8-b47b-aac728ac4b2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:07.394283 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d14e6-2b52-45d8-b47b-aac728ac4b2e service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:07.394865 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d14e6-2b52-45d8-b47b-aac728ac4b2e service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:07.395381 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d14e6-2b52-45d8-b47b-aac728ac4b2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:07.422196 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d14e6-2b52-45d8-b47b-aac728ac4b2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:07.423875 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d14e6-2b52-45d8-b47b-aac728ac4b2e service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:07.440771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 843/1700] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:07.447343 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853e7e5-202f-492c-a02b-e4a39e8a1b66 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:07.447808 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853e7e5-202f-492c-a02b-e4a39e8a1b66 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:07.449058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853e7e5-202f-492c-a02b-e4a39e8a1b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:07.529945 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6853e7e5-202f-492c-a02b-e4a39e8a1b66 None None] MFA Rules not processed for user `5aa7ff2fa5454bba9268e89c44b75de6`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:07.532600 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6853e7e5-202f-492c-a02b-e4a39e8a1b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:07.733499 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 858/1701] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:07.740923 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7603cbb-4c7e-4ec9-8d05-c36ac6c11de7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:07.741537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7603cbb-4c7e-4ec9-8d05-c36ac6c11de7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:07.742022 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7603cbb-4c7e-4ec9-8d05-c36ac6c11de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:07.774341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-e7603cbb-4c7e-4ec9-8d05-c36ac6c11de7 None None] MFA Rules not processed for user `5aa7ff2fa5454bba9268e89c44b75de6`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:07.776003 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e7603cbb-4c7e-4ec9-8d05-c36ac6c11de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:07.785183 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 844/1702] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:07.798488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853e7e5-202f-492c-a02b-e4a39e8a1b66 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:07.800182 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6853e7e5-202f-492c-a02b-e4a39e8a1b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:07.828288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853e7e5-202f-492c-a02b-e4a39e8a1b66 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:07.845548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e269cf-1a31-45f0-9057-09d313f37cdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:07.846400 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e269cf-1a31-45f0-9057-09d313f37cdb service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:07.846770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e269cf-1a31-45f0-9057-09d313f37cdb service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:07.847614 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e269cf-1a31-45f0-9057-09d313f37cdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:07.858744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e269cf-1a31-45f0-9057-09d313f37cdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:07.860795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e269cf-1a31-45f0-9057-09d313f37cdb service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:07.880423 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 859/1703] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:07 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:08.097406 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755ce45-0a15-481d-9292-043ca6b9fdb4 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:08.097887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755ce45-0a15-481d-9292-043ca6b9fdb4 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:08.098482 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755ce45-0a15-481d-9292-043ca6b9fdb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:08.138812 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f755ce45-0a15-481d-9292-043ca6b9fdb4 None None] MFA Rules not processed for user `b72067d66a7649e7b8c120dd4d3fe635`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:08.140479 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f755ce45-0a15-481d-9292-043ca6b9fdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:08.182418 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e269cf-1a31-45f0-9057-09d313f37cdb service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:08.184157 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-65e269cf-1a31-45f0-9057-09d313f37cdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:08.271158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:08.285617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:08.365818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e269cf-1a31-45f0-9057-09d313f37cdb service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:08.371026 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 845/1704] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:08.372293 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755ce45-0a15-481d-9292-043ca6b9fdb4 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:08.379924 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f755ce45-0a15-481d-9292-043ca6b9fdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:08.383263 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82c0d0-7668-489a-b505-9b553ef0127a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:08.385407 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82c0d0-7668-489a-b505-9b553ef0127a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:08.385702 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82c0d0-7668-489a-b505-9b553ef0127a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:08.391505 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82c0d0-7668-489a-b505-9b553ef0127a admin admin] PATH_INFO: `/v3/projects/1050d9df0b3a4888bc6c1a7299fd4b7a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:08.393942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc82c0d0-7668-489a-b505-9b553ef0127a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1050d9df0b3a4888bc6c1a7299fd4b7a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:08.401918 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc82c0d0-7668-489a-b505-9b553ef0127a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:08.461823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755ce45-0a15-481d-9292-043ca6b9fdb4 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:08.476793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ed6d2-aebe-4a57-bfd7-adebfd83c63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:08.485303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ed6d2-aebe-4a57-bfd7-adebfd83c63d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:08.488194 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ed6d2-aebe-4a57-bfd7-adebfd83c63d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:08.488707 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ed6d2-aebe-4a57-bfd7-adebfd83c63d admin admin] PATH_INFO: `/v3/projects/249678c9fcd64c478017877a077720ee` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:08.490506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ed6d2-aebe-4a57-bfd7-adebfd83c63d admin admin] RBAC: Authorizing `identity:delete_project(project_id=249678c9fcd64c478017877a077720ee)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:08.494354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ed6d2-aebe-4a57-bfd7-adebfd83c63d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:08.626205 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bc82c0d0-7668-489a-b505-9b553ef0127a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1050d9df0b3a4888bc6c1a7299fd4b7a', 'request_id': 'req-bc82c0d0-7668-489a-b505-9b553ef0127a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:08.726624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-cc4ed6d2-aebe-4a57-bfd7-adebfd83c63d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '249678c9fcd64c478017877a077720ee', 'request_id': 'req-cc4ed6d2-aebe-4a57-bfd7-adebfd83c63d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:08.746464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bc82c0d0-7668-489a-b505-9b553ef0127a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1050d9df0b3a4888bc6c1a7299fd4b7a', 'request_id': 'req-bc82c0d0-7668-489a-b505-9b553ef0127a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:08.754677 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bc82c0d0-7668-489a-b505-9b553ef0127a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1050d9df0b3a4888bc6c1a7299fd4b7a', 'request_id': 'req-bc82c0d0-7668-489a-b505-9b553ef0127a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:08.790139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bc82c0d0-7668-489a-b505-9b553ef0127a admin admin] The token cache is being invalidate because project 1050d9df0b3a4888bc6c1a7299fd4b7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:08:08.790139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bc82c0d0-7668-489a-b505-9b553ef0127a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:08.790139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bc82c0d0-7668-489a-b505-9b553ef0127a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:08.790139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 860/1705] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:08:08 2022] DELETE /identity/v3/projects/1050d9df0b3a4888bc6c1a7299fd4b7a => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:08:08.790139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82c0d0-7668-489a-b505-9b553ef0127a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:08.791407 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bc82c0d0-7668-489a-b505-9b553ef0127a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:08.890944 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-cc4ed6d2-aebe-4a57-bfd7-adebfd83c63d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '249678c9fcd64c478017877a077720ee', 'request_id': 'req-cc4ed6d2-aebe-4a57-bfd7-adebfd83c63d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:08.905602 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-cc4ed6d2-aebe-4a57-bfd7-adebfd83c63d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '249678c9fcd64c478017877a077720ee', 'request_id': 'req-cc4ed6d2-aebe-4a57-bfd7-adebfd83c63d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:08.950340 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-cc4ed6d2-aebe-4a57-bfd7-adebfd83c63d admin admin] The token cache is being invalidate because project 249678c9fcd64c478017877a077720ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:08:08.950740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-cc4ed6d2-aebe-4a57-bfd7-adebfd83c63d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:08.951860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-cc4ed6d2-aebe-4a57-bfd7-adebfd83c63d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:08.955673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 846/1706] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:08:08 2022] DELETE /identity/v3/projects/249678c9fcd64c478017877a077720ee => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:08.958118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82c0d0-7668-489a-b505-9b553ef0127a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:08.958594 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54cadf-944d-4015-b942-b22261457e4e None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:08.958901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54cadf-944d-4015-b942-b22261457e4e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:08.969674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54cadf-944d-4015-b942-b22261457e4e None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:08.969674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 847/1707] 10.222.0.69 () {54 vars in 912 bytes} [Thu Jul 14 19:08:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:08:08.969674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66c75f-a7bf-41f4-9365-9f0400279dc7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:08.970041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66c75f-a7bf-41f4-9365-9f0400279dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:08.970335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66c75f-a7bf-41f4-9365-9f0400279dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:09.024825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-cb66c75f-a7bf-41f4-9365-9f0400279dc7 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:09.026829 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-cb66c75f-a7bf-41f4-9365-9f0400279dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:09.080941 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:09.130175 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4892364-9cac-4fa6-8488-b6b73f7fc01e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:09.130838 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4892364-9cac-4fa6-8488-b6b73f7fc01e service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:09.131783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4892364-9cac-4fa6-8488-b6b73f7fc01e service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:09.132068 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4892364-9cac-4fa6-8488-b6b73f7fc01e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:09.134241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b4892364-9cac-4fa6-8488-b6b73f7fc01e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:09.198696 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 848/1708] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:09.205650 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66c75f-a7bf-41f4-9365-9f0400279dc7 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:09.210400 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-cb66c75f-a7bf-41f4-9365-9f0400279dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:09.244916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66c75f-a7bf-41f4-9365-9f0400279dc7 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:09.253867 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07178feb-f7e6-4208-bd60-b46297705c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:09.255643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07178feb-f7e6-4208-bd60-b46297705c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:09.256624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07178feb-f7e6-4208-bd60-b46297705c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:09.257071 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07178feb-f7e6-4208-bd60-b46297705c47 admin admin] PATH_INFO: `/v3/users/e0c759aaf5244c7fb85d2df571d9e349` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:09.258376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4892364-9cac-4fa6-8488-b6b73f7fc01e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:09.260512 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07178feb-f7e6-4208-bd60-b46297705c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0c759aaf5244c7fb85d2df571d9e349)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:09.261602 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4892364-9cac-4fa6-8488-b6b73f7fc01e service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:09.263140 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07178feb-f7e6-4208-bd60-b46297705c47 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:09.281843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 861/1709] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:09.283301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c4729-5571-4011-bd7a-a30e8af79d76 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:09.283602 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c4729-5571-4011-bd7a-a30e8af79d76 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:09.283885 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c4729-5571-4011-bd7a-a30e8af79d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:09.329494 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-8e3c4729-5571-4011-bd7a-a30e8af79d76 None None] MFA Rules not processed for user `74b8c75b9cce442b8c430fab6a3fea17`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:09.330712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8e3c4729-5571-4011-bd7a-a30e8af79d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:09.411115 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 862/1710] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 14 19:08:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:09.413530 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41df366-1705-4792-adfc-7b6ebb086bcc None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:09.415402 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41df366-1705-4792-adfc-7b6ebb086bcc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:09.415854 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41df366-1705-4792-adfc-7b6ebb086bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:09.442639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-b41df366-1705-4792-adfc-7b6ebb086bcc None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:09.444747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b41df366-1705-4792-adfc-7b6ebb086bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:09.463061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 863/1711] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:09.467746 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58666f0b-8581-464c-b425-94e03931825b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:09.471783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58666f0b-8581-464c-b425-94e03931825b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:09.471783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58666f0b-8581-464c-b425-94e03931825b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:09.530432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-58666f0b-8581-464c-b425-94e03931825b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:09.533819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-07178feb-f7e6-4208-bd60-b46297705c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0c759aaf5244c7fb85d2df571d9e349'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:09.534299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-58666f0b-8581-464c-b425-94e03931825b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:09.555625 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-07178feb-f7e6-4208-bd60-b46297705c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0c759aaf5244c7fb85d2df571d9e349'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:09.557183 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-07178feb-f7e6-4208-bd60-b46297705c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0c759aaf5244c7fb85d2df571d9e349'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:09.558932 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-07178feb-f7e6-4208-bd60-b46297705c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0c759aaf5244c7fb85d2df571d9e349'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:09.559595 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 864/1712] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:09.563555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58666f0b-8581-464c-b425-94e03931825b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:09.566806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-58666f0b-8581-464c-b425-94e03931825b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:09.594457 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58666f0b-8581-464c-b425-94e03931825b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:09.599021 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f5dbf-af1c-4bff-ab39-15934edba000 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:09.599771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f5dbf-af1c-4bff-ab39-15934edba000 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:09.600057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f5dbf-af1c-4bff-ab39-15934edba000 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:09.600334 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f5dbf-af1c-4bff-ab39-15934edba000 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:09.603293 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0a0f5dbf-af1c-4bff-ab39-15934edba000 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:09.611899 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-07178feb-f7e6-4208-bd60-b46297705c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0c759aaf5244c7fb85d2df571d9e349'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:09.620494 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f5dbf-af1c-4bff-ab39-15934edba000 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:09.625831 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f5dbf-af1c-4bff-ab39-15934edba000 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:09.650274 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 865/1713] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:09.654589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e9efe-ddc5-4283-8943-ba92d4e2fb30 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:09.655006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e9efe-ddc5-4283-8943-ba92d4e2fb30 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:09.656086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e9efe-ddc5-4283-8943-ba92d4e2fb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:09.712828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 849/1714] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:08:09 2022] DELETE /identity/v3/users/e0c759aaf5244c7fb85d2df571d9e349 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:09.714074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07178feb-f7e6-4208-bd60-b46297705c47 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:09.719255 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-07178feb-f7e6-4208-bd60-b46297705c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:09.730972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-883e9efe-ddc5-4283-8943-ba92d4e2fb30 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:09.737658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-883e9efe-ddc5-4283-8943-ba92d4e2fb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:09.983143 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07178feb-f7e6-4208-bd60-b46297705c47 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:09.998278 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481b313-b8ab-4442-8a0a-a01f0a42ac91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:09.998965 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481b313-b8ab-4442-8a0a-a01f0a42ac91 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:09.999439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 866/1715] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:10.001331 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481b313-b8ab-4442-8a0a-a01f0a42ac91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:10.001926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481b313-b8ab-4442-8a0a-a01f0a42ac91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:10.002419 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b46cab-ca43-4756-a21d-74aca1a89306 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:10.003359 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481b313-b8ab-4442-8a0a-a01f0a42ac91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:08:10.006959 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b46cab-ca43-4756-a21d-74aca1a89306 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:10.006959 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b46cab-ca43-4756-a21d-74aca1a89306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:10.007311 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481b313-b8ab-4442-8a0a-a01f0a42ac91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:10.009900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481b313-b8ab-4442-8a0a-a01f0a42ac91 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:10.046801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 850/1716] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:08:10.047961 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481b313-b8ab-4442-8a0a-a01f0a42ac91 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:10.070895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-82b46cab-ca43-4756-a21d-74aca1a89306 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:10.072181 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-82b46cab-ca43-4756-a21d-74aca1a89306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:10.087087 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481b313-b8ab-4442-8a0a-a01f0a42ac91 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:10.092642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 867/1717] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:10.107269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b46cab-ca43-4756-a21d-74aca1a89306 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:10.111110 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01338958-af21-4507-8fb6-ec309da60f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:10.111842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01338958-af21-4507-8fb6-ec309da60f9e admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:10.112139 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01338958-af21-4507-8fb6-ec309da60f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:10.112424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01338958-af21-4507-8fb6-ec309da60f9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:10.113176 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01338958-af21-4507-8fb6-ec309da60f9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:08:10.113495 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01338958-af21-4507-8fb6-ec309da60f9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:10.115147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01338958-af21-4507-8fb6-ec309da60f9e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:10.128337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b46cab-ca43-4756-a21d-74aca1a89306 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:10.154540 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e92d4-ec91-4d14-82a8-15ca37290107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:10.155634 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e92d4-ec91-4d14-82a8-15ca37290107 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:10.155997 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e92d4-ec91-4d14-82a8-15ca37290107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:10.156296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e92d4-ec91-4d14-82a8-15ca37290107 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:10.157023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e92d4-ec91-4d14-82a8-15ca37290107 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:10.159445 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e92d4-ec91-4d14-82a8-15ca37290107 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:10.165793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 851/1718] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:08:10.167753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01338958-af21-4507-8fb6-ec309da60f9e admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:10.189908 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01338958-af21-4507-8fb6-ec309da60f9e admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:10.204013 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e922c8e-61f7-4135-bcbc-5c4eef248bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:10.205149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e922c8e-61f7-4135-bcbc-5c4eef248bcb admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:10.205438 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e922c8e-61f7-4135-bcbc-5c4eef248bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:10.207849 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e922c8e-61f7-4135-bcbc-5c4eef248bcb admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:10.209704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e922c8e-61f7-4135-bcbc-5c4eef248bcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:10.216555 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e922c8e-61f7-4135-bcbc-5c4eef248bcb admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:10.329026 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 868/1719] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:08:10 2022] POST /identity/v3/projects => generated 373 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:10.332199 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e92d4-ec91-4d14-82a8-15ca37290107 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:10.342337 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:10.344144 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-844e92d4-ec91-4d14-82a8-15ca37290107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:10.371727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 852/1720] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:08:10 2022] POST /identity/v3/projects => generated 377 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:10.376889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e922c8e-61f7-4135-bcbc-5c4eef248bcb admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:10.508215 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e92d4-ec91-4d14-82a8-15ca37290107 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:10.583429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac469d56-862a-49d4-ad18-62dfe28f4fb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:10.589909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac469d56-862a-49d4-ad18-62dfe28f4fb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:10.589909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac469d56-862a-49d4-ad18-62dfe28f4fb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:10.589909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac469d56-862a-49d4-ad18-62dfe28f4fb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:10.604097 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e922c8e-61f7-4135-bcbc-5c4eef248bcb admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:10.610585 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac469d56-862a-49d4-ad18-62dfe28f4fb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:10.614472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac469d56-862a-49d4-ad18-62dfe28f4fb8 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:10.621117 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1229e5c-ba52-4604-bc84-25536b714c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:10.622116 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1229e5c-ba52-4604-bc84-25536b714c03 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:10.623329 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1229e5c-ba52-4604-bc84-25536b714c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:10.623695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1229e5c-ba52-4604-bc84-25536b714c03 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:10.625982 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1229e5c-ba52-4604-bc84-25536b714c03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:10.631720 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1229e5c-ba52-4604-bc84-25536b714c03 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:10.642675 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 869/1721] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:10.644319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac469d56-862a-49d4-ad18-62dfe28f4fb8 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:10.665401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac469d56-862a-49d4-ad18-62dfe28f4fb8 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:10.672621 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ef142-6d96-4ec9-bc6c-a98a16da461f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:10.673761 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ef142-6d96-4ec9-bc6c-a98a16da461f admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:10.674354 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ef142-6d96-4ec9-bc6c-a98a16da461f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:10.674932 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ef142-6d96-4ec9-bc6c-a98a16da461f admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:10.676049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ef142-6d96-4ec9-bc6c-a98a16da461f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:10.681557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ef142-6d96-4ec9-bc6c-a98a16da461f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:10.806933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 853/1723] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:08:10 2022] POST /identity/v3/users => generated 287 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:10.807226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 870/1723] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:08:10 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:10.810628 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1229e5c-ba52-4604-bc84-25536b714c03 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:10.812341 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51201b-1555-4064-b5ac-68168e32f804 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:10.813083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51201b-1555-4064-b5ac-68168e32f804 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:10.815686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51201b-1555-4064-b5ac-68168e32f804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:10.855597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1229e5c-ba52-4604-bc84-25536b714c03 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:10.871587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7bf65-b540-47ad-a086-6e4f0df72b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:10.876025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7bf65-b540-47ad-a086-6e4f0df72b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:10.876025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7bf65-b540-47ad-a086-6e4f0df72b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:10.876025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7bf65-b540-47ad-a086-6e4f0df72b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:10.877146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7bf65-b540-47ad-a086-6e4f0df72b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:10.880737 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7bf65-b540-47ad-a086-6e4f0df72b64 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:10.933235 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 854/1724] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:10.943656 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7bf65-b540-47ad-a086-6e4f0df72b64 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:10.962946 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7bf65-b540-47ad-a086-6e4f0df72b64 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:10.985373 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d7900-df4c-4137-acfb-80b37b84e6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:10.985373 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d7900-df4c-4137-acfb-80b37b84e6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:10.985373 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d7900-df4c-4137-acfb-80b37b84e6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:10.985373 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d7900-df4c-4137-acfb-80b37b84e6d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:10.985735 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d7900-df4c-4137-acfb-80b37b84e6d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:10.991154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d7900-df4c-4137-acfb-80b37b84e6d2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:11.017835 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 855/1725] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:11.019245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d7900-df4c-4137-acfb-80b37b84e6d2 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:11.035365 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6c51201b-1555-4064-b5ac-68168e32f804 None None] MFA Rules not processed for user `4ef0fad9a71d43bfbaba9c87dfed41c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:11.036506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6c51201b-1555-4064-b5ac-68168e32f804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:11.057678 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d7900-df4c-4137-acfb-80b37b84e6d2 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:11.077117 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fc825-8dfb-484b-8453-b9257560aa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:11.077834 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fc825-8dfb-484b-8453-b9257560aa70 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:11.078121 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fc825-8dfb-484b-8453-b9257560aa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:11.078410 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fc825-8dfb-484b-8453-b9257560aa70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:11.082369 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fc825-8dfb-484b-8453-b9257560aa70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:11.086370 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fc825-8dfb-484b-8453-b9257560aa70 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:11.128853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 856/1726] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:11.130988 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fc825-8dfb-484b-8453-b9257560aa70 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:11.173556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fc825-8dfb-484b-8453-b9257560aa70 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:11.189184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0ba34-7794-4552-be9f-c886f29a877f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:11.189969 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0ba34-7794-4552-be9f-c886f29a877f admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:11.190291 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0ba34-7794-4552-be9f-c886f29a877f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:11.190580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0ba34-7794-4552-be9f-c886f29a877f admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:11.191638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0ba34-7794-4552-be9f-c886f29a877f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:11.193287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0ba34-7794-4552-be9f-c886f29a877f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:11.214945 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:11.231055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 857/1727] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:11.235222 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0ba34-7794-4552-be9f-c886f29a877f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:11.274773 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0ba34-7794-4552-be9f-c886f29a877f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:11.292925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac2d3d-23d7-4421-87cb-d1748359208a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:11.299562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac2d3d-23d7-4421-87cb-d1748359208a admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:11.299562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac2d3d-23d7-4421-87cb-d1748359208a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:11.299562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac2d3d-23d7-4421-87cb-d1748359208a admin admin] PATH_INFO: `/v3/projects/fbaf9a08ae8146c8a4c3e0d9cc932fa2/users/808af6661f374669ad036c5120d1ac93/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:11.312843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 871/1728] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 14 19:08:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:11.314061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51201b-1555-4064-b5ac-68168e32f804 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:11.365793 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51201b-1555-4064-b5ac-68168e32f804 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:11.371040 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac2d3d-23d7-4421-87cb-d1748359208a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbaf9a08ae8146c8a4c3e0d9cc932fa2, user_id=808af6661f374669ad036c5120d1ac93, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:11.374206 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db640400-60b3-4a29-89ff-de13f46f1949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:11.375018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac2d3d-23d7-4421-87cb-d1748359208a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:11.375865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db640400-60b3-4a29-89ff-de13f46f1949 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:11.376349 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db640400-60b3-4a29-89ff-de13f46f1949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:11.376803 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db640400-60b3-4a29-89ff-de13f46f1949 admin admin] PATH_INFO: `/v3/projects/40d521bd6dbd470f9739c7d5c40863b8/users/cd25ff075bc244b799bebac3a2889b65/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:11.425318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db640400-60b3-4a29-89ff-de13f46f1949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d521bd6dbd470f9739c7d5c40863b8, user_id=cd25ff075bc244b799bebac3a2889b65, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:11.439831 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db640400-60b3-4a29-89ff-de13f46f1949 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:11.540648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 858/1729] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:11 2022] PUT /identity/v3/projects/fbaf9a08ae8146c8a4c3e0d9cc932fa2/users/808af6661f374669ad036c5120d1ac93/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:11.540648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac2d3d-23d7-4421-87cb-d1748359208a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:11.543599 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0aac2d3d-23d7-4421-87cb-d1748359208a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:11.592690 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 872/1730] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:11 2022] PUT /identity/v3/projects/40d521bd6dbd470f9739c7d5c40863b8/users/cd25ff075bc244b799bebac3a2889b65/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:11.592690 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dff148-c3e6-4f97-9e27-bc09b3f7071b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:11.592690 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dff148-c3e6-4f97-9e27-bc09b3f7071b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:11.592690 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dff148-c3e6-4f97-9e27-bc09b3f7071b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:11.695927 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-80dff148-c3e6-4f97-9e27-bc09b3f7071b None None] MFA Rules not processed for user `808af6661f374669ad036c5120d1ac93`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:11.706356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-80dff148-c3e6-4f97-9e27-bc09b3f7071b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:11.748086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:11.748501 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac2d3d-23d7-4421-87cb-d1748359208a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:11.940070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec878d52-bbe0-4cb1-8b99-93a9f7756e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:11.941219 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec878d52-bbe0-4cb1-8b99-93a9f7756e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:11.946716 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec878d52-bbe0-4cb1-8b99-93a9f7756e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:11.950861 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec878d52-bbe0-4cb1-8b99-93a9f7756e20 admin admin] PATH_INFO: `/v3/users/934dc97df7434ace9c19757a9cd8cf18` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:11.964163 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec878d52-bbe0-4cb1-8b99-93a9f7756e20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=934dc97df7434ace9c19757a9cd8cf18)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:11.970198 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 873/1731] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:11.971509 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac399f-ce14-4be5-b9ae-92f7ebe59169 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:11.971806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac399f-ce14-4be5-b9ae-92f7ebe59169 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:11.972093 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac399f-ce14-4be5-b9ae-92f7ebe59169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:11.980341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec878d52-bbe0-4cb1-8b99-93a9f7756e20 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:12.066109 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ecac399f-ce14-4be5-b9ae-92f7ebe59169 None None] MFA Rules not processed for user `cd25ff075bc244b799bebac3a2889b65`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:12.067734 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ecac399f-ce14-4be5-b9ae-92f7ebe59169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:12.266760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 874/1732] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:12.268373 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ece8c2-8acf-4f3c-bd40-9c1e1e3515f7 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:12.268848 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ece8c2-8acf-4f3c-bd40-9c1e1e3515f7 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:12.269356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ece8c2-8acf-4f3c-bd40-9c1e1e3515f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:12.276957 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ec878d52-bbe0-4cb1-8b99-93a9f7756e20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '934dc97df7434ace9c19757a9cd8cf18'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:12.289983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-82ece8c2-8acf-4f3c-bd40-9c1e1e3515f7 None None] MFA Rules not processed for user `5aa7ff2fa5454bba9268e89c44b75de6`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:12.292595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ec878d52-bbe0-4cb1-8b99-93a9f7756e20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '934dc97df7434ace9c19757a9cd8cf18'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:12.292595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ec878d52-bbe0-4cb1-8b99-93a9f7756e20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '934dc97df7434ace9c19757a9cd8cf18'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:12.292812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-82ece8c2-8acf-4f3c-bd40-9c1e1e3515f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:12.292915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ec878d52-bbe0-4cb1-8b99-93a9f7756e20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '934dc97df7434ace9c19757a9cd8cf18'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:12.317877 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ec878d52-bbe0-4cb1-8b99-93a9f7756e20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '934dc97df7434ace9c19757a9cd8cf18'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:12.428990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 875/1733] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:12.428990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ece8c2-8acf-4f3c-bd40-9c1e1e3515f7 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:12.430881 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-82ece8c2-8acf-4f3c-bd40-9c1e1e3515f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:12.439679 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 859/1734] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:08:11 2022] DELETE /identity/v3/users/934dc97df7434ace9c19757a9cd8cf18 => generated 0 bytes in 914 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:12.441483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec878d52-bbe0-4cb1-8b99-93a9f7756e20 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:12.444292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ec878d52-bbe0-4cb1-8b99-93a9f7756e20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:12.496241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:12.624887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec878d52-bbe0-4cb1-8b99-93a9f7756e20 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:12.639792 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73116a9f-f20c-445f-b4b4-acf307337fb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:12.641689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73116a9f-f20c-445f-b4b4-acf307337fb4 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:12.642917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73116a9f-f20c-445f-b4b4-acf307337fb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:12.644226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73116a9f-f20c-445f-b4b4-acf307337fb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:12.649510 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-73116a9f-f20c-445f-b4b4-acf307337fb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:12.656168 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ece8c2-8acf-4f3c-bd40-9c1e1e3515f7 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:12.769617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73116a9f-f20c-445f-b4b4-acf307337fb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:12.775515 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03677431-ac39-41b4-abe2-a0f7239d212d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4368f68724f946eca53084a9ec7ff590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:12.776609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73116a9f-f20c-445f-b4b4-acf307337fb4 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:12.777675 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03677431-ac39-41b4-abe2-a0f7239d212d service swift] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:12.778785 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03677431-ac39-41b4-abe2-a0f7239d212d service swift] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:12.779348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03677431-ac39-41b4-abe2-a0f7239d212d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:12.784856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-03677431-ac39-41b4-abe2-a0f7239d212d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:12.801057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 860/1735] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:12.802641 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73116a9f-f20c-445f-b4b4-acf307337fb4 service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:12.806010 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-73116a9f-f20c-445f-b4b4-acf307337fb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:12.959758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03677431-ac39-41b4-abe2-a0f7239d212d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:12.964492 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03677431-ac39-41b4-abe2-a0f7239d212d service swift] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:12.990345 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 876/1736] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 14 19:08:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 19:08:12.992440 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f2de4-5fbf-42c9-a474-8cbb2344693b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:13.046944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f2de4-5fbf-42c9-a474-8cbb2344693b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:13.046944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f2de4-5fbf-42c9-a474-8cbb2344693b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:13.047966 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73116a9f-f20c-445f-b4b4-acf307337fb4 service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:13.047966 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c4a2b-2a97-4d42-b1bc-df09f0739a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:13.047966 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c4a2b-2a97-4d42-b1bc-df09f0739a12 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:13.047966 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c4a2b-2a97-4d42-b1bc-df09f0739a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:13.048292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c4a2b-2a97-4d42-b1bc-df09f0739a12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:13.048292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c4a2b-2a97-4d42-b1bc-df09f0739a12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:13.048292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c4a2b-2a97-4d42-b1bc-df09f0739a12 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:13.048520 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-623f2de4-5fbf-42c9-a474-8cbb2344693b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:13.072008 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-623f2de4-5fbf-42c9-a474-8cbb2344693b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:13.094418 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 877/1737] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:13.119058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f2de4-5fbf-42c9-a474-8cbb2344693b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:13.120927 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-623f2de4-5fbf-42c9-a474-8cbb2344693b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:13.209481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 861/1738] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:08:12 2022] POST /identity/v3/projects => generated 375 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:13.214159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c4a2b-2a97-4d42-b1bc-df09f0739a12 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:13.337560 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f2de4-5fbf-42c9-a474-8cbb2344693b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:13.355797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:13.415038 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c4a2b-2a97-4d42-b1bc-df09f0739a12 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:13.430616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c40620-635b-45e6-8e13-375ea1f500a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:13.430616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c40620-635b-45e6-8e13-375ea1f500a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:13.430616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c40620-635b-45e6-8e13-375ea1f500a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:13.430616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c40620-635b-45e6-8e13-375ea1f500a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:13.431450 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c40620-635b-45e6-8e13-375ea1f500a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:13.438066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c40620-635b-45e6-8e13-375ea1f500a4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:13.518647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a48b1-8232-448c-a052-5c013196ea22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:13.519400 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a48b1-8232-448c-a052-5c013196ea22 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:13.519705 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a48b1-8232-448c-a052-5c013196ea22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:13.519992 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a48b1-8232-448c-a052-5c013196ea22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:13.542656 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a48b1-8232-448c-a052-5c013196ea22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:13.543947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a48b1-8232-448c-a052-5c013196ea22 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:13.576247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 878/1739] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:08:13.576247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1be2cf-f914-4a32-ad99-505e6b1df7cf None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:13.578783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1be2cf-f914-4a32-ad99-505e6b1df7cf None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:13.579122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1be2cf-f914-4a32-ad99-505e6b1df7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:13.634701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 862/1740] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:08:13 2022] POST /identity/v3/users => generated 294 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:13.634701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c40620-635b-45e6-8e13-375ea1f500a4 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:13.634701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b3c40620-635b-45e6-8e13-375ea1f500a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:13.635436 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-bf1be2cf-f914-4a32-ad99-505e6b1df7cf None None] MFA Rules not processed for user `74b8c75b9cce442b8c430fab6a3fea17`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:13.635436 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bf1be2cf-f914-4a32-ad99-505e6b1df7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:13.646256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 879/1741] 10.222.0.30 () {58 vars in 1015 bytes} [Thu Jul 14 19:08:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:13.647579 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1be2cf-f914-4a32-ad99-505e6b1df7cf None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:13.655075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c40620-635b-45e6-8e13-375ea1f500a4 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:13.676545 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1be2cf-f914-4a32-ad99-505e6b1df7cf None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:13.683053 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662782d-955a-4ba8-84b0-243b7d1cbce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:13.683787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662782d-955a-4ba8-84b0-243b7d1cbce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:13.684095 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662782d-955a-4ba8-84b0-243b7d1cbce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:13.684354 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662782d-955a-4ba8-84b0-243b7d1cbce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:13.685197 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662782d-955a-4ba8-84b0-243b7d1cbce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:13.685985 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab570b-9469-4905-8888-74b45e7210bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:13.686822 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662782d-955a-4ba8-84b0-243b7d1cbce2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:13.687083 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab570b-9469-4905-8888-74b45e7210bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:13.687692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab570b-9469-4905-8888-74b45e7210bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:13.688057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab570b-9469-4905-8888-74b45e7210bf admin admin] PATH_INFO: `/v3/projects/65d1be152dc44aafb1178d275e152aec` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:13.696605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab570b-9469-4905-8888-74b45e7210bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=65d1be152dc44aafb1178d275e152aec)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:13.701587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab570b-9469-4905-8888-74b45e7210bf admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:13.714956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 880/1742] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:13.714956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662782d-955a-4ba8-84b0-243b7d1cbce2 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:13.750648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662782d-955a-4ba8-84b0-243b7d1cbce2 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:13.765372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29a6f0-c8b0-483c-ab38-ffe182d20bca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:13.766246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29a6f0-c8b0-483c-ab38-ffe182d20bca service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:13.766363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29a6f0-c8b0-483c-ab38-ffe182d20bca service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:13.766585 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29a6f0-c8b0-483c-ab38-ffe182d20bca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:13.789244 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29a6f0-c8b0-483c-ab38-ffe182d20bca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:13.790796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29a6f0-c8b0-483c-ab38-ffe182d20bca service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:13.805129 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 881/1743] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:13.806053 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29a6f0-c8b0-483c-ab38-ffe182d20bca service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:13.845018 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29a6f0-c8b0-483c-ab38-ffe182d20bca service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:13.873739 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc531573-8bf2-4bb1-94a5-890404e474f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:13.874657 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc531573-8bf2-4bb1-94a5-890404e474f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:13.874943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc531573-8bf2-4bb1-94a5-890404e474f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:13.875261 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc531573-8bf2-4bb1-94a5-890404e474f9 admin admin] PATH_INFO: `/v3/projects/872dd756982b4d9585b456641b67d12b/users/553b440942d345478336133e77492bb4/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:13.924809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-62ab570b-9469-4905-8888-74b45e7210bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65d1be152dc44aafb1178d275e152aec', 'request_id': 'req-62ab570b-9469-4905-8888-74b45e7210bf'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:13.928268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc531573-8bf2-4bb1-94a5-890404e474f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=872dd756982b4d9585b456641b67d12b, user_id=553b440942d345478336133e77492bb4, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:13.931497 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc531573-8bf2-4bb1-94a5-890404e474f9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:14.050274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-62ab570b-9469-4905-8888-74b45e7210bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65d1be152dc44aafb1178d275e152aec', 'request_id': 'req-62ab570b-9469-4905-8888-74b45e7210bf'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:14.060167 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 882/1744] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:13 2022] PUT /identity/v3/projects/872dd756982b4d9585b456641b67d12b/users/553b440942d345478336133e77492bb4/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:14.063509 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc531573-8bf2-4bb1-94a5-890404e474f9 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:14.065985 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-62ab570b-9469-4905-8888-74b45e7210bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65d1be152dc44aafb1178d275e152aec', 'request_id': 'req-62ab570b-9469-4905-8888-74b45e7210bf'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:14.088564 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-62ab570b-9469-4905-8888-74b45e7210bf admin admin] The token cache is being invalidate because project 65d1be152dc44aafb1178d275e152aec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:08:14.098695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-62ab570b-9469-4905-8888-74b45e7210bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:14.098695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-62ab570b-9469-4905-8888-74b45e7210bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:14.098695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 863/1745] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:08:13 2022] DELETE /identity/v3/projects/65d1be152dc44aafb1178d275e152aec => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:14.100421 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab570b-9469-4905-8888-74b45e7210bf admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:14.102087 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-62ab570b-9469-4905-8888-74b45e7210bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:14.225553 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc531573-8bf2-4bb1-94a5-890404e474f9 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:14.259947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bd20d-a429-4dab-b6c0-6446e2dfa225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:14.261048 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bd20d-a429-4dab-b6c0-6446e2dfa225 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:14.264258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bd20d-a429-4dab-b6c0-6446e2dfa225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:14.264764 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bd20d-a429-4dab-b6c0-6446e2dfa225 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:14.266374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2bd20d-a429-4dab-b6c0-6446e2dfa225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:14.268225 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2bd20d-a429-4dab-b6c0-6446e2dfa225 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:14.307807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 883/1746] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:14.308959 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bd20d-a429-4dab-b6c0-6446e2dfa225 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:14.319976 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0d2bd20d-a429-4dab-b6c0-6446e2dfa225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:14.393928 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bd20d-a429-4dab-b6c0-6446e2dfa225 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:14.423846 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26ce61-401c-4fac-93c5-37a2feb81795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:14.424822 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26ce61-401c-4fac-93c5-37a2feb81795 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:14.425121 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26ce61-401c-4fac-93c5-37a2feb81795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:14.425427 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26ce61-401c-4fac-93c5-37a2feb81795 admin admin] PATH_INFO: `/v3/projects/872dd756982b4d9585b456641b67d12b/users/553b440942d345478336133e77492bb4/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:14.431681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c26ce61-401c-4fac-93c5-37a2feb81795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=872dd756982b4d9585b456641b67d12b, user_id=553b440942d345478336133e77492bb4, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:14.446504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab570b-9469-4905-8888-74b45e7210bf admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:14.450628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c26ce61-401c-4fac-93c5-37a2feb81795 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:14.523033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2f10a-bd6f-4e23-b205-c9064ea59d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:14.523033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2f10a-bd6f-4e23-b205-c9064ea59d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:14.523033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2f10a-bd6f-4e23-b205-c9064ea59d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:14.527775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2f10a-bd6f-4e23-b205-c9064ea59d42 admin admin] PATH_INFO: `/v3/users/491d2a0af36e473d84d5d54e2acba117` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:14.538217 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2f10a-bd6f-4e23-b205-c9064ea59d42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=491d2a0af36e473d84d5d54e2acba117)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:14.554267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2f10a-bd6f-4e23-b205-c9064ea59d42 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:14.555145 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 884/1747] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:14 2022] PUT /identity/v3/projects/872dd756982b4d9585b456641b67d12b/users/553b440942d345478336133e77492bb4/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:14.589853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faeb085-b6d7-4a4d-83ce-7bf90159c16b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:14.589853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faeb085-b6d7-4a4d-83ce-7bf90159c16b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:14.589853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faeb085-b6d7-4a4d-83ce-7bf90159c16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:14.853441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3faeb085-b6d7-4a4d-83ce-7bf90159c16b None None] MFA Rules not processed for user `553b440942d345478336133e77492bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:14.857409 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3faeb085-b6d7-4a4d-83ce-7bf90159c16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:14.975083 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:15.022277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3cf2f10a-bd6f-4e23-b205-c9064ea59d42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '491d2a0af36e473d84d5d54e2acba117'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:15.085443 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3cf2f10a-bd6f-4e23-b205-c9064ea59d42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '491d2a0af36e473d84d5d54e2acba117'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:15.088017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3cf2f10a-bd6f-4e23-b205-c9064ea59d42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '491d2a0af36e473d84d5d54e2acba117'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:15.092159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3cf2f10a-bd6f-4e23-b205-c9064ea59d42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '491d2a0af36e473d84d5d54e2acba117'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:15.174055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3cf2f10a-bd6f-4e23-b205-c9064ea59d42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '491d2a0af36e473d84d5d54e2acba117'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:15.175049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:15.254165 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 885/1748] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 667 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:15.258856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9b02f-83ac-416f-b8a5-8133bc821e30 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:15.259665 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9b02f-83ac-416f-b8a5-8133bc821e30 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:15.262926 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9b02f-83ac-416f-b8a5-8133bc821e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:15.283302 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 864/1749] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:08:14 2022] DELETE /identity/v3/users/491d2a0af36e473d84d5d54e2acba117 => generated 0 bytes in 1183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:15.287044 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2f10a-bd6f-4e23-b205-c9064ea59d42 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:15.288176 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3cf2f10a-bd6f-4e23-b205-c9064ea59d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:15.329789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-70a9b02f-83ac-416f-b8a5-8133bc821e30 None None] MFA Rules not processed for user `7d5fa180ee554e8b8e46321892969489`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:15.331722 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-70a9b02f-83ac-416f-b8a5-8133bc821e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:15.477491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2f10a-bd6f-4e23-b205-c9064ea59d42 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:15.515157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23eaff-d6cc-40a3-b2d2-4d29bf5a12ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:15.515157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23eaff-d6cc-40a3-b2d2-4d29bf5a12ab service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:15.515157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23eaff-d6cc-40a3-b2d2-4d29bf5a12ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:15.515157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23eaff-d6cc-40a3-b2d2-4d29bf5a12ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:15.515769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-bd23eaff-d6cc-40a3-b2d2-4d29bf5a12ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:15.515843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 886/1750] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:15.696254 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23eaff-d6cc-40a3-b2d2-4d29bf5a12ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:15.701895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23eaff-d6cc-40a3-b2d2-4d29bf5a12ab service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:15.739834 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 865/1751] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:15.934888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:16.669933 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9b02f-83ac-416f-b8a5-8133bc821e30 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:16.677237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-70a9b02f-83ac-416f-b8a5-8133bc821e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:16.872503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9b02f-83ac-416f-b8a5-8133bc821e30 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:16.878787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039106fc-ee5b-4eb4-8b11-34cc901c0234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:16.881332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039106fc-ee5b-4eb4-8b11-34cc901c0234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:16.881735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039106fc-ee5b-4eb4-8b11-34cc901c0234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:16.882466 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039106fc-ee5b-4eb4-8b11-34cc901c0234 admin admin] PATH_INFO: `/v3/projects/6daccaf10bc341268a69861e3b23206b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:16.884858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039106fc-ee5b-4eb4-8b11-34cc901c0234 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6daccaf10bc341268a69861e3b23206b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:16.888400 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039106fc-ee5b-4eb4-8b11-34cc901c0234 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:17.416875 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23eaff-d6cc-40a3-b2d2-4d29bf5a12ab service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:17.420395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-bd23eaff-d6cc-40a3-b2d2-4d29bf5a12ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:17.439445 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-039106fc-ee5b-4eb4-8b11-34cc901c0234 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6daccaf10bc341268a69861e3b23206b', 'request_id': 'req-039106fc-ee5b-4eb4-8b11-34cc901c0234'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:17.443248 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23eaff-d6cc-40a3-b2d2-4d29bf5a12ab service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:17.598065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-039106fc-ee5b-4eb4-8b11-34cc901c0234 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6daccaf10bc341268a69861e3b23206b', 'request_id': 'req-039106fc-ee5b-4eb4-8b11-34cc901c0234'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:17.606734 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-039106fc-ee5b-4eb4-8b11-34cc901c0234 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6daccaf10bc341268a69861e3b23206b', 'request_id': 'req-039106fc-ee5b-4eb4-8b11-34cc901c0234'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:17.635781 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-039106fc-ee5b-4eb4-8b11-34cc901c0234 admin admin] The token cache is being invalidate because project 6daccaf10bc341268a69861e3b23206b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:08:17.636302 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-039106fc-ee5b-4eb4-8b11-34cc901c0234 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:17.640033 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-039106fc-ee5b-4eb4-8b11-34cc901c0234 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:17.643215 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 887/1752] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:08:16 2022] DELETE /identity/v3/projects/6daccaf10bc341268a69861e3b23206b => generated 0 bytes in 974 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:17.647797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594a059-adbd-468a-9572-ae87d5c7afd2 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:17.648255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594a059-adbd-468a-9572-ae87d5c7afd2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:17.648834 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594a059-adbd-468a-9572-ae87d5c7afd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:17.649209 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca94254-c580-4f9d-bd85-ff9e2f6e5f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:17.650012 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca94254-c580-4f9d-bd85-ff9e2f6e5f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:17.650299 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca94254-c580-4f9d-bd85-ff9e2f6e5f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:17.650610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca94254-c580-4f9d-bd85-ff9e2f6e5f14 admin admin] PATH_INFO: `/v3/projects/9f089badaa0b48988c62a394a7a4b179` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:17.653928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca94254-c580-4f9d-bd85-ff9e2f6e5f14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f089badaa0b48988c62a394a7a4b179)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:17.666428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca94254-c580-4f9d-bd85-ff9e2f6e5f14 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:17.678513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6594a059-adbd-468a-9572-ae87d5c7afd2 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:17.680117 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6594a059-adbd-468a-9572-ae87d5c7afd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:17.695261 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 888/1753] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:17.702880 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ffbf3-d1ec-47dc-a2d2-2034091de857 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:17.704303 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ffbf3-d1ec-47dc-a2d2-2034091de857 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:17.707228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ffbf3-d1ec-47dc-a2d2-2034091de857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:17.763373 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-772ffbf3-d1ec-47dc-a2d2-2034091de857 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:17.763373 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-772ffbf3-d1ec-47dc-a2d2-2034091de857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:17.782189 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:17.799026 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 889/1754] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:08:17.803603 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ffbf3-d1ec-47dc-a2d2-2034091de857 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:17.815997 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ffbf3-d1ec-47dc-a2d2-2034091de857 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:17.828238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdf753-c012-44b4-a261-fc8abc2afcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:17.829854 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdf753-c012-44b4-a261-fc8abc2afcef admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:17.830173 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdf753-c012-44b4-a261-fc8abc2afcef admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:17.830439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdf753-c012-44b4-a261-fc8abc2afcef admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:17.831202 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbdf753-c012-44b4-a261-fc8abc2afcef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:08:17.833122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbdf753-c012-44b4-a261-fc8abc2afcef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:17.839631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbdf753-c012-44b4-a261-fc8abc2afcef admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:17.874801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 890/1755] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 19:08:17.877265 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdf753-c012-44b4-a261-fc8abc2afcef admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:17.920026 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:17.942924 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-cca94254-c580-4f9d-bd85-ff9e2f6e5f14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f089badaa0b48988c62a394a7a4b179', 'request_id': 'req-cca94254-c580-4f9d-bd85-ff9e2f6e5f14'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:17.950284 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdf753-c012-44b4-a261-fc8abc2afcef admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:18.071535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-cca94254-c580-4f9d-bd85-ff9e2f6e5f14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f089badaa0b48988c62a394a7a4b179', 'request_id': 'req-cca94254-c580-4f9d-bd85-ff9e2f6e5f14'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:18.081424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-cca94254-c580-4f9d-bd85-ff9e2f6e5f14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f089badaa0b48988c62a394a7a4b179', 'request_id': 'req-cca94254-c580-4f9d-bd85-ff9e2f6e5f14'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:18.124041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-cca94254-c580-4f9d-bd85-ff9e2f6e5f14 admin admin] The token cache is being invalidate because project 9f089badaa0b48988c62a394a7a4b179 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:08:18.124393 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-cca94254-c580-4f9d-bd85-ff9e2f6e5f14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:18.129891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-cca94254-c580-4f9d-bd85-ff9e2f6e5f14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:18.139658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79498e-cdb1-4edf-91ec-b53a79400276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:18.140218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 866/1756] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:08:17 2022] DELETE /identity/v3/projects/9f089badaa0b48988c62a394a7a4b179 => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:08:18.141246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79498e-cdb1-4edf-91ec-b53a79400276 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:18.141702 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79498e-cdb1-4edf-91ec-b53a79400276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:18.142143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79498e-cdb1-4edf-91ec-b53a79400276 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:18.143017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79498e-cdb1-4edf-91ec-b53a79400276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:18.145895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79498e-cdb1-4edf-91ec-b53a79400276 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:18.152928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca94254-c580-4f9d-bd85-ff9e2f6e5f14 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:18.168366 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-cca94254-c580-4f9d-bd85-ff9e2f6e5f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:18.205865 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca94254-c580-4f9d-bd85-ff9e2f6e5f14 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:18.235785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dc9cc-d74d-47ad-ae53-2ebe7f763371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:18.236236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dc9cc-d74d-47ad-ae53-2ebe7f763371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:18.236528 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dc9cc-d74d-47ad-ae53-2ebe7f763371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:18.236984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dc9cc-d74d-47ad-ae53-2ebe7f763371 admin admin] PATH_INFO: `/v3/users/808af6661f374669ad036c5120d1ac93` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:18.239973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89dc9cc-d74d-47ad-ae53-2ebe7f763371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=808af6661f374669ad036c5120d1ac93)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:18.244767 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89dc9cc-d74d-47ad-ae53-2ebe7f763371 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:18.279021 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 891/1757] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:08:17 2022] POST /identity/v3/projects => generated 369 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:18.281317 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fe765-091f-4fd4-a366-09606427d56e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:18.281773 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fe765-091f-4fd4-a366-09606427d56e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:18.282506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fe765-091f-4fd4-a366-09606427d56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:18.336511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-518fe765-091f-4fd4-a366-09606427d56e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:18.337780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-518fe765-091f-4fd4-a366-09606427d56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:18.531107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f89dc9cc-d74d-47ad-ae53-2ebe7f763371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '808af6661f374669ad036c5120d1ac93'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:18.549739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f89dc9cc-d74d-47ad-ae53-2ebe7f763371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '808af6661f374669ad036c5120d1ac93'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:18.552915 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 892/1758] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:18.554669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f89dc9cc-d74d-47ad-ae53-2ebe7f763371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '808af6661f374669ad036c5120d1ac93'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:18.555630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fe765-091f-4fd4-a366-09606427d56e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:18.557865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-518fe765-091f-4fd4-a366-09606427d56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:18.558245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f89dc9cc-d74d-47ad-ae53-2ebe7f763371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '808af6661f374669ad036c5120d1ac93'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:18.598064 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f89dc9cc-d74d-47ad-ae53-2ebe7f763371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '808af6661f374669ad036c5120d1ac93'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:18.606839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fe765-091f-4fd4-a366-09606427d56e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:18.626434 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756cd1c-2a09-4c31-b8f1-062ad1384409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:18.627379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756cd1c-2a09-4c31-b8f1-062ad1384409 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:18.634798 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756cd1c-2a09-4c31-b8f1-062ad1384409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:18.635745 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756cd1c-2a09-4c31-b8f1-062ad1384409 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:18.641414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756cd1c-2a09-4c31-b8f1-062ad1384409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:18.648367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756cd1c-2a09-4c31-b8f1-062ad1384409 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:18.685400 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 867/1759] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:08:18 2022] DELETE /identity/v3/users/808af6661f374669ad036c5120d1ac93 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:18.685400 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6d766-2425-42d5-8358-86bcfa0226eb None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:18.685400 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6d766-2425-42d5-8358-86bcfa0226eb None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:18.685400 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6d766-2425-42d5-8358-86bcfa0226eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:18.736848 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7dc6d766-2425-42d5-8358-86bcfa0226eb None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:18.738126 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7dc6d766-2425-42d5-8358-86bcfa0226eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:18.797745 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 893/1760] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:08:18 2022] POST /identity/v3/users => generated 285 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:18.800665 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cdf67-cbaf-436f-9607-946477ac8ef3 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:18.804096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cdf67-cbaf-436f-9607-946477ac8ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:18.804853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cdf67-cbaf-436f-9607-946477ac8ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:18.910635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-bd8cdf67-cbaf-436f-9607-946477ac8ef3 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:18.914045 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bd8cdf67-cbaf-436f-9607-946477ac8ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:18.937515 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 868/1761] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:18.938971 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6d766-2425-42d5-8358-86bcfa0226eb None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:18.942272 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7dc6d766-2425-42d5-8358-86bcfa0226eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:18.956946 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 894/1762] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:18.957916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cdf67-cbaf-436f-9607-946477ac8ef3 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:18.993854 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cdf67-cbaf-436f-9607-946477ac8ef3 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:18.994039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6d766-2425-42d5-8358-86bcfa0226eb None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:19.009457 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14396f7-2ef6-48a0-80c7-bbf27685f152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:19.009591 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bd0e2-dc80-43be-ae23-c94d4eb6dffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:19.010452 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bd0e2-dc80-43be-ae23-c94d4eb6dffc admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:19.010752 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bd0e2-dc80-43be-ae23-c94d4eb6dffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:19.011036 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bd0e2-dc80-43be-ae23-c94d4eb6dffc admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:19.011677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14396f7-2ef6-48a0-80c7-bbf27685f152 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:19.017783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bd0e2-dc80-43be-ae23-c94d4eb6dffc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:08:19.017783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bd0e2-dc80-43be-ae23-c94d4eb6dffc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:19.018006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14396f7-2ef6-48a0-80c7-bbf27685f152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:19.018096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bd0e2-dc80-43be-ae23-c94d4eb6dffc admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:19.018205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14396f7-2ef6-48a0-80c7-bbf27685f152 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:19.018205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14396f7-2ef6-48a0-80c7-bbf27685f152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:19.020935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14396f7-2ef6-48a0-80c7-bbf27685f152 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:19.056373 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 869/1763] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:19.060717 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14396f7-2ef6-48a0-80c7-bbf27685f152 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:19.088438 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14396f7-2ef6-48a0-80c7-bbf27685f152 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:19.105418 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6689255-7b15-44df-8de0-c60b8a067e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:19.109904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6689255-7b15-44df-8de0-c60b8a067e8d admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:19.109904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6689255-7b15-44df-8de0-c60b8a067e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:19.109904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6689255-7b15-44df-8de0-c60b8a067e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:19.112532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 895/1764] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:08:19.113534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6689255-7b15-44df-8de0-c60b8a067e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:19.113656 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bd0e2-dc80-43be-ae23-c94d4eb6dffc admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:19.119676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bd3bd0e2-dc80-43be-ae23-c94d4eb6dffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:19.123785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6689255-7b15-44df-8de0-c60b8a067e8d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:19.187542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 870/1765] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:08:19.196224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6689255-7b15-44df-8de0-c60b8a067e8d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:19.223925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6689255-7b15-44df-8de0-c60b8a067e8d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:19.248504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab9382-14cb-4595-9ac4-d1b1dd046422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:19.249218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab9382-14cb-4595-9ac4-d1b1dd046422 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:19.251493 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab9382-14cb-4595-9ac4-d1b1dd046422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:19.251493 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab9382-14cb-4595-9ac4-d1b1dd046422 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:19.258043 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab9382-14cb-4595-9ac4-d1b1dd046422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:19.263773 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab9382-14cb-4595-9ac4-d1b1dd046422 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:19.286968 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bd0e2-dc80-43be-ae23-c94d4eb6dffc admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:19.307152 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0ee08-67ad-4d61-9660-ceee3a7cb795 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:19.307928 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0ee08-67ad-4d61-9660-ceee3a7cb795 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:19.311746 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0ee08-67ad-4d61-9660-ceee3a7cb795 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:19.312248 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0ee08-67ad-4d61-9660-ceee3a7cb795 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:19.390575 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0ee08-67ad-4d61-9660-ceee3a7cb795 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:19.398177 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0ee08-67ad-4d61-9660-ceee3a7cb795 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:19.459955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:19.460827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 871/1766] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:08:19 2022] POST /identity/v3/projects => generated 371 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:19.461753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab9382-14cb-4595-9ac4-d1b1dd046422 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:19.468983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 896/1767] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:19.472052 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df19ee-f539-4362-9ed5-4b7c3e252a5d None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:19.482345 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df19ee-f539-4362-9ed5-4b7c3e252a5d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:19.482345 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df19ee-f539-4362-9ed5-4b7c3e252a5d None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:19.482345 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 897/1768] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 14 19:08:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:08:19.484479 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df19ee-f539-4362-9ed5-4b7c3e252a5d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:19.776235 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:19.790820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab9382-14cb-4595-9ac4-d1b1dd046422 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:19.803906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df19ee-f539-4362-9ed5-4b7c3e252a5d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:19.808249 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d0893-1fb5-40b6-9cb1-7eb93f8d1d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:19.815464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d0893-1fb5-40b6-9cb1-7eb93f8d1d13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:19.816722 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d0893-1fb5-40b6-9cb1-7eb93f8d1d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:19.817849 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d0893-1fb5-40b6-9cb1-7eb93f8d1d13 admin admin] PATH_INFO: `/v3/projects/50e1644b846a4f5aa76e84fac98cd25e/users/392898e8ef8446e9938e205f5cbd92d5/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:19.822866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e98a98-3753-4cd2-b94b-77ad2b84f318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:19.827040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e98a98-3753-4cd2-b94b-77ad2b84f318 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:19.827040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e98a98-3753-4cd2-b94b-77ad2b84f318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:19.838645 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e98a98-3753-4cd2-b94b-77ad2b84f318 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:19.839912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e98a98-3753-4cd2-b94b-77ad2b84f318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:19.849493 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e98a98-3753-4cd2-b94b-77ad2b84f318 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:19.867586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d0893-1fb5-40b6-9cb1-7eb93f8d1d13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50e1644b846a4f5aa76e84fac98cd25e, user_id=392898e8ef8446e9938e205f5cbd92d5, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:19.870810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d0893-1fb5-40b6-9cb1-7eb93f8d1d13 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:19.967333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 898/1769] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:08:19 2022] POST /identity/v3/users => generated 286 bytes in 487 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:19.970833 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f682968-2c80-48be-8589-0b6216443627 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:19.971244 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f682968-2c80-48be-8589-0b6216443627 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:19.971566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f682968-2c80-48be-8589-0b6216443627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:19.980233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 872/1770] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:19 2022] PUT /identity/v3/projects/50e1644b846a4f5aa76e84fac98cd25e/users/392898e8ef8446e9938e205f5cbd92d5/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:19.983980 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c2186-6986-42e0-a173-b7ff1c4b2626 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:19.985916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c2186-6986-42e0-a173-b7ff1c4b2626 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:19.986215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c2186-6986-42e0-a173-b7ff1c4b2626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:20.055405 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9f682968-2c80-48be-8589-0b6216443627 None None] MFA Rules not processed for user `b72067d66a7649e7b8c120dd4d3fe635`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:20.056953 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9f682968-2c80-48be-8589-0b6216443627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:20.057663 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-164c2186-6986-42e0-a173-b7ff1c4b2626 None None] MFA Rules not processed for user `8d27d6df5cb64697adebd961dfbec324`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:20.064033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-164c2186-6986-42e0-a173-b7ff1c4b2626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:20.188855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 899/1771] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:20.189889 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f682968-2c80-48be-8589-0b6216443627 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:20.257457 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 873/1772] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 14 19:08:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:20.260634 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3fec9-a164-4c52-b07d-1f4a076356e9 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:20.263479 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3fec9-a164-4c52-b07d-1f4a076356e9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:20.265267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3fec9-a164-4c52-b07d-1f4a076356e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:20.390129 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d2e3fec9-a164-4c52-b07d-1f4a076356e9 None None] MFA Rules not processed for user `392898e8ef8446e9938e205f5cbd92d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:20.392896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d2e3fec9-a164-4c52-b07d-1f4a076356e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:20.408374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f682968-2c80-48be-8589-0b6216443627 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:20.422039 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4d4da-f90c-40a4-835c-0fc3dfc7e4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:20.422892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4d4da-f90c-40a4-835c-0fc3dfc7e4fa admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:20.423205 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4d4da-f90c-40a4-835c-0fc3dfc7e4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:20.423562 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4d4da-f90c-40a4-835c-0fc3dfc7e4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:20.424470 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4d4da-f90c-40a4-835c-0fc3dfc7e4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:20.426322 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4d4da-f90c-40a4-835c-0fc3dfc7e4fa admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:20.467517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 900/1773] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:20.468558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4d4da-f90c-40a4-835c-0fc3dfc7e4fa admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:20.470575 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-38f4d4da-f90c-40a4-835c-0fc3dfc7e4fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:20.504577 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4d4da-f90c-40a4-835c-0fc3dfc7e4fa admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:20.526368 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8eb3c-0bf2-4edd-b581-04850674c2f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:20.527126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8eb3c-0bf2-4edd-b581-04850674c2f4 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:20.527487 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8eb3c-0bf2-4edd-b581-04850674c2f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:20.527831 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8eb3c-0bf2-4edd-b581-04850674c2f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:20.551245 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8eb3c-0bf2-4edd-b581-04850674c2f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:20.552546 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 874/1774] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:20 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:20.553051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8eb3c-0bf2-4edd-b581-04850674c2f4 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:20.554703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3fec9-a164-4c52-b07d-1f4a076356e9 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:20.581214 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 901/1775] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:20.582501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8eb3c-0bf2-4edd-b581-04850674c2f4 service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:20.583985 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3fec9-a164-4c52-b07d-1f4a076356e9 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:20.599063 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13661da0-3c05-4225-a665-d5423c08db64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:20.602238 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13661da0-3c05-4225-a665-d5423c08db64 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:20.602399 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13661da0-3c05-4225-a665-d5423c08db64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:20.602871 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13661da0-3c05-4225-a665-d5423c08db64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:20.605058 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13661da0-3c05-4225-a665-d5423c08db64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:20.607396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8eb3c-0bf2-4edd-b581-04850674c2f4 service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:20.610648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13661da0-3c05-4225-a665-d5423c08db64 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:20.625514 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae85157-91c2-476b-8c0e-377995bd97b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:20.626324 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae85157-91c2-476b-8c0e-377995bd97b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:20.626648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae85157-91c2-476b-8c0e-377995bd97b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:20.627109 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae85157-91c2-476b-8c0e-377995bd97b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:20.627999 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae85157-91c2-476b-8c0e-377995bd97b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:20.630171 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae85157-91c2-476b-8c0e-377995bd97b2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:20.635920 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 875/1776] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:20.643843 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13661da0-3c05-4225-a665-d5423c08db64 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:20.677441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13661da0-3c05-4225-a665-d5423c08db64 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:20.688636 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a760398-421f-40b4-ab2f-e6fa7c0c6a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:20.689774 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a760398-421f-40b4-ab2f-e6fa7c0c6a0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:20.690070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a760398-421f-40b4-ab2f-e6fa7c0c6a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:20.690357 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a760398-421f-40b4-ab2f-e6fa7c0c6a0d admin admin] PATH_INFO: `/v3/projects/cc29a1f5e0054fc0976a1059f0fb760b/users/cdd54a12768648ceb46e68bbe626606f/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:20.726200 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a760398-421f-40b4-ab2f-e6fa7c0c6a0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc29a1f5e0054fc0976a1059f0fb760b, user_id=cdd54a12768648ceb46e68bbe626606f, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:20.729358 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a760398-421f-40b4-ab2f-e6fa7c0c6a0d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:20.788718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 902/1777] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:08:20 2022] POST /identity/v3/projects => generated 369 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:20.793733 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae85157-91c2-476b-8c0e-377995bd97b2 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:20.849724 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:20.851541 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 876/1778] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:20 2022] PUT /identity/v3/projects/cc29a1f5e0054fc0976a1059f0fb760b/users/cdd54a12768648ceb46e68bbe626606f/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:20.864926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9b54b-ac15-487c-8990-8ecc2643c2b7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:20.871721 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9b54b-ac15-487c-8990-8ecc2643c2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:20.871721 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9b54b-ac15-487c-8990-8ecc2643c2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:20.946356 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae85157-91c2-476b-8c0e-377995bd97b2 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:20.966740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-44a9b54b-ac15-487c-8990-8ecc2643c2b7 None None] MFA Rules not processed for user `cdd54a12768648ceb46e68bbe626606f`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:20.968082 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-44a9b54b-ac15-487c-8990-8ecc2643c2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:21.111646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:21.120204 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 877/1779] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:21.121084 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172d170-fc3d-44fb-8278-ff7f57668bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:21.122636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172d170-fc3d-44fb-8278-ff7f57668bfb admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:21.125952 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9b54b-ac15-487c-8990-8ecc2643c2b7 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:21.126747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172d170-fc3d-44fb-8278-ff7f57668bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:21.129147 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172d170-fc3d-44fb-8278-ff7f57668bfb admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:21.131668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172d170-fc3d-44fb-8278-ff7f57668bfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:21.137118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172d170-fc3d-44fb-8278-ff7f57668bfb admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:21.140448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9b54b-ac15-487c-8990-8ecc2643c2b7 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:21.146375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4be66-d67e-4cb4-8820-4038cc2196b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:21.147657 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4be66-d67e-4cb4-8820-4038cc2196b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:21.148043 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4be66-d67e-4cb4-8820-4038cc2196b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:21.148388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4be66-d67e-4cb4-8820-4038cc2196b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:21.149509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4be66-d67e-4cb4-8820-4038cc2196b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:21.160215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4be66-d67e-4cb4-8820-4038cc2196b6 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:21.274998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 903/1780] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:08:20 2022] POST /identity/v3/users => generated 291 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:08:21.279082 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172d170-fc3d-44fb-8278-ff7f57668bfb admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:21.306495 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 878/1781] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:08:21 2022] POST /identity/v3/projects => generated 373 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:21.307547 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4be66-d67e-4cb4-8820-4038cc2196b6 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:21.312433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172d170-fc3d-44fb-8278-ff7f57668bfb admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:21.433162 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40586ce-2cca-480f-9797-9293205d3892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:21.434877 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40586ce-2cca-480f-9797-9293205d3892 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:21.435193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40586ce-2cca-480f-9797-9293205d3892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:21.435494 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40586ce-2cca-480f-9797-9293205d3892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:21.445420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40586ce-2cca-480f-9797-9293205d3892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:21.485163 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40586ce-2cca-480f-9797-9293205d3892 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:21.485163 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 904/1782] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:21 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:21.487227 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4be66-d67e-4cb4-8820-4038cc2196b6 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:21.492217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40586ce-2cca-480f-9797-9293205d3892 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:21.509010 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e2bfa-ebd9-4900-a422-82ad0fab314d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:21.510103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e2bfa-ebd9-4900-a422-82ad0fab314d admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:21.510439 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e2bfa-ebd9-4900-a422-82ad0fab314d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:21.513154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40586ce-2cca-480f-9797-9293205d3892 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:21.517052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e2bfa-ebd9-4900-a422-82ad0fab314d admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:21.517921 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e2bfa-ebd9-4900-a422-82ad0fab314d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:21.521447 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e2bfa-ebd9-4900-a422-82ad0fab314d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:21.526290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286372c0-86f0-4c39-a887-6be51c15a109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:21.527279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286372c0-86f0-4c39-a887-6be51c15a109 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:21.527579 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286372c0-86f0-4c39-a887-6be51c15a109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:21.527869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286372c0-86f0-4c39-a887-6be51c15a109 admin admin] PATH_INFO: `/v3/projects/4f10ac6940af45859e2606ba086e7f7d/users/7aa5fbf6170d40ab85f394a213e66c85/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:21.552287 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286372c0-86f0-4c39-a887-6be51c15a109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f10ac6940af45859e2606ba086e7f7d, user_id=7aa5fbf6170d40ab85f394a213e66c85, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:21.556266 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286372c0-86f0-4c39-a887-6be51c15a109 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:22.027377 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 905/1783] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:21 2022] PUT /identity/v3/projects/4f10ac6940af45859e2606ba086e7f7d/users/7aa5fbf6170d40ab85f394a213e66c85/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:22.030912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286372c0-86f0-4c39-a887-6be51c15a109 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:22.034567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 879/1784] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:08:21 2022] POST /identity/v3/users => generated 293 bytes in 724 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:22.034567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e2bfa-ebd9-4900-a422-82ad0fab314d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:22.038179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-286372c0-86f0-4c39-a887-6be51c15a109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:22.061922 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e2bfa-ebd9-4900-a422-82ad0fab314d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:22.074977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:22.159072 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65159528-4be7-4488-b00a-534ae4566d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:22.159872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65159528-4be7-4488-b00a-534ae4566d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:22.160096 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65159528-4be7-4488-b00a-534ae4566d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:22.160373 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65159528-4be7-4488-b00a-534ae4566d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:22.161113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65159528-4be7-4488-b00a-534ae4566d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:22.163292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65159528-4be7-4488-b00a-534ae4566d72 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:22.180669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 880/1785] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:22.181844 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65159528-4be7-4488-b00a-534ae4566d72 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:22.210101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65159528-4be7-4488-b00a-534ae4566d72 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:22.210686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286372c0-86f0-4c39-a887-6be51c15a109 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:22.220854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832d671-d614-4d9c-9a5d-a744c9138302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:22.221265 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832d671-d614-4d9c-9a5d-a744c9138302 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:22.221559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832d671-d614-4d9c-9a5d-a744c9138302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:22.221834 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832d671-d614-4d9c-9a5d-a744c9138302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:22.227963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b832d671-d614-4d9c-9a5d-a744c9138302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:22.230786 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b832d671-d614-4d9c-9a5d-a744c9138302 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:22.232689 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0ed7c-408b-43a7-bd40-b4516d66b74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:22.234534 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0ed7c-408b-43a7-bd40-b4516d66b74d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:22.236220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0ed7c-408b-43a7-bd40-b4516d66b74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:22.237150 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0ed7c-408b-43a7-bd40-b4516d66b74d admin admin] PATH_INFO: `/v3/projects/fbaf9a08ae8146c8a4c3e0d9cc932fa2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:22.242539 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0ed7c-408b-43a7-bd40-b4516d66b74d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbaf9a08ae8146c8a4c3e0d9cc932fa2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:22.243130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 881/1786] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:22.245457 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0ed7c-408b-43a7-bd40-b4516d66b74d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:22.252094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832d671-d614-4d9c-9a5d-a744c9138302 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:22.283235 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832d671-d614-4d9c-9a5d-a744c9138302 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:22.288644 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d75b0-428e-4966-8051-91f5bec918af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:22.289620 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d75b0-428e-4966-8051-91f5bec918af admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:22.290153 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d75b0-428e-4966-8051-91f5bec918af admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:22.290626 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d75b0-428e-4966-8051-91f5bec918af admin admin] PATH_INFO: `/v3/projects/4f10ac6940af45859e2606ba086e7f7d/users/7aa5fbf6170d40ab85f394a213e66c85/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:22.295723 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d75b0-428e-4966-8051-91f5bec918af admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f10ac6940af45859e2606ba086e7f7d, user_id=7aa5fbf6170d40ab85f394a213e66c85, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:22.298827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d75b0-428e-4966-8051-91f5bec918af admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:22.324614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 882/1787] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:22 2022] PUT /identity/v3/projects/4f10ac6940af45859e2606ba086e7f7d/users/7aa5fbf6170d40ab85f394a213e66c85/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:22.325853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d75b0-428e-4966-8051-91f5bec918af admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:22.420984 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fcf0ed7c-408b-43a7-bd40-b4516d66b74d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbaf9a08ae8146c8a4c3e0d9cc932fa2', 'request_id': 'req-fcf0ed7c-408b-43a7-bd40-b4516d66b74d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:22.457343 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d75b0-428e-4966-8051-91f5bec918af admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:22.524254 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fcf0ed7c-408b-43a7-bd40-b4516d66b74d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbaf9a08ae8146c8a4c3e0d9cc932fa2', 'request_id': 'req-fcf0ed7c-408b-43a7-bd40-b4516d66b74d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:22.537375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fcf0ed7c-408b-43a7-bd40-b4516d66b74d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbaf9a08ae8146c8a4c3e0d9cc932fa2', 'request_id': 'req-fcf0ed7c-408b-43a7-bd40-b4516d66b74d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:22.560532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fcf0ed7c-408b-43a7-bd40-b4516d66b74d admin admin] The token cache is being invalidate because project fbaf9a08ae8146c8a4c3e0d9cc932fa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:08:22.567667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fcf0ed7c-408b-43a7-bd40-b4516d66b74d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:22.567667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fcf0ed7c-408b-43a7-bd40-b4516d66b74d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:22.567667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 906/1788] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:08:22 2022] DELETE /identity/v3/projects/fbaf9a08ae8146c8a4c3e0d9cc932fa2 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:22.568364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e225bb2-e776-4387-acf6-a3f94612125f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:22.568835 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e225bb2-e776-4387-acf6-a3f94612125f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:22.569279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e225bb2-e776-4387-acf6-a3f94612125f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:22.571704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa51f2-ce2b-49d9-b454-647595db5749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:22.572882 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa51f2-ce2b-49d9-b454-647595db5749 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:22.573580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa51f2-ce2b-49d9-b454-647595db5749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:22.574095 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa51f2-ce2b-49d9-b454-647595db5749 admin admin] PATH_INFO: `/v3/projects/7cb97d9c20294d9caeb09b5ecc111b87/users/d780c67ddb274103afe60e7a8837390d/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:22.595601 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa51f2-ce2b-49d9-b454-647595db5749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cb97d9c20294d9caeb09b5ecc111b87, user_id=d780c67ddb274103afe60e7a8837390d, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:22.598861 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa51f2-ce2b-49d9-b454-647595db5749 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:22.612337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-8e225bb2-e776-4387-acf6-a3f94612125f None None] MFA Rules not processed for user `7aa5fbf6170d40ab85f394a213e66c85`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:22.615908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8e225bb2-e776-4387-acf6-a3f94612125f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:22.666152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 883/1789] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:22 2022] PUT /identity/v3/projects/7cb97d9c20294d9caeb09b5ecc111b87/users/d780c67ddb274103afe60e7a8837390d/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:22.667584 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2340edf-c49c-49e9-ac48-a39f31caf19e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:22.667878 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2340edf-c49c-49e9-ac48-a39f31caf19e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:22.668190 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2340edf-c49c-49e9-ac48-a39f31caf19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:22.679177 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:22.693289 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-e2340edf-c49c-49e9-ac48-a39f31caf19e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:22.694461 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e2340edf-c49c-49e9-ac48-a39f31caf19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:22.723802 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 907/1790] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:22 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:22.724756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e225bb2-e776-4387-acf6-a3f94612125f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:22.726266 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8e225bb2-e776-4387-acf6-a3f94612125f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:22.741040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e225bb2-e776-4387-acf6-a3f94612125f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:22.780842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 884/1791] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:22.780842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2340edf-c49c-49e9-ac48-a39f31caf19e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:22.782439 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e2340edf-c49c-49e9-ac48-a39f31caf19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:22.824908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378491eb-6609-462e-91ea-a135d88e2cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:22.826986 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378491eb-6609-462e-91ea-a135d88e2cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:22.827782 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378491eb-6609-462e-91ea-a135d88e2cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:22.828223 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378491eb-6609-462e-91ea-a135d88e2cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:22.829129 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378491eb-6609-462e-91ea-a135d88e2cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:22.831014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378491eb-6609-462e-91ea-a135d88e2cc1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:22.842831 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 908/1792] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:22.844148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535912df-751d-4ea7-830b-b1d43c554af5 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:22.844489 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535912df-751d-4ea7-830b-b1d43c554af5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:22.844918 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535912df-751d-4ea7-830b-b1d43c554af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:22.866186 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-535912df-751d-4ea7-830b-b1d43c554af5 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:22.866678 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2340edf-c49c-49e9-ac48-a39f31caf19e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:22.867621 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-535912df-751d-4ea7-830b-b1d43c554af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:22.871022 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aed02d-7970-4d1b-bae1-4cd077d7a949 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:22.873228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aed02d-7970-4d1b-bae1-4cd077d7a949 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:22.873228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aed02d-7970-4d1b-bae1-4cd077d7a949 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:22.873228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aed02d-7970-4d1b-bae1-4cd077d7a949 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:22.878088 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 909/1793] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:22.879680 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535912df-751d-4ea7-830b-b1d43c554af5 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:22.893007 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535912df-751d-4ea7-830b-b1d43c554af5 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:22.898836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c73723-328f-4bbd-8c92-0db84c4e8a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:22.900007 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c73723-328f-4bbd-8c92-0db84c4e8a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:22.900295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c73723-328f-4bbd-8c92-0db84c4e8a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:22.900580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c73723-328f-4bbd-8c92-0db84c4e8a5b admin admin] PATH_INFO: `/v3/projects/7cb97d9c20294d9caeb09b5ecc111b87/users/d780c67ddb274103afe60e7a8837390d/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:22.902834 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c73723-328f-4bbd-8c92-0db84c4e8a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cb97d9c20294d9caeb09b5ecc111b87, user_id=d780c67ddb274103afe60e7a8837390d, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:22.905719 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c73723-328f-4bbd-8c92-0db84c4e8a5b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:22.919218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 910/1794] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:22 2022] PUT /identity/v3/projects/7cb97d9c20294d9caeb09b5ecc111b87/users/d780c67ddb274103afe60e7a8837390d/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:22.920540 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c73723-328f-4bbd-8c92-0db84c4e8a5b admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:22.946215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aed02d-7970-4d1b-bae1-4cd077d7a949 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:22.947715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aed02d-7970-4d1b-bae1-4cd077d7a949 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:22.961245 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:22.976009 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 885/1795] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:22 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:22.977358 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616891f-3452-4237-825e-0e3df295ff85 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:22.977655 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616891f-3452-4237-825e-0e3df295ff85 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:22.978011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616891f-3452-4237-825e-0e3df295ff85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:23.033844 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-6616891f-3452-4237-825e-0e3df295ff85 None None] MFA Rules not processed for user `d780c67ddb274103afe60e7a8837390d`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:23.034449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c73723-328f-4bbd-8c92-0db84c4e8a5b admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:23.039332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6616891f-3452-4237-825e-0e3df295ff85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:23.041169 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b34d3d-3306-4c80-b178-dc56148d2105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:23.041992 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b34d3d-3306-4c80-b178-dc56148d2105 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:23.042215 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b34d3d-3306-4c80-b178-dc56148d2105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:23.042768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b34d3d-3306-4c80-b178-dc56148d2105 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:23.043577 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b34d3d-3306-4c80-b178-dc56148d2105 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:08:23.044379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b34d3d-3306-4c80-b178-dc56148d2105 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:23.045552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b34d3d-3306-4c80-b178-dc56148d2105 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:23.070315 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 911/1796] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:08:23.070315 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b34d3d-3306-4c80-b178-dc56148d2105 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:23.083017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b34d3d-3306-4c80-b178-dc56148d2105 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:23.092800 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94a62b-9217-4061-b55a-b18377699288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:23.092800 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94a62b-9217-4061-b55a-b18377699288 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:23.092800 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94a62b-9217-4061-b55a-b18377699288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:23.092800 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94a62b-9217-4061-b55a-b18377699288 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:23.093723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94a62b-9217-4061-b55a-b18377699288 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:23.097255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94a62b-9217-4061-b55a-b18377699288 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:23.164635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 886/1797] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:23.164635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616891f-3452-4237-825e-0e3df295ff85 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:23.166555 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6616891f-3452-4237-825e-0e3df295ff85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:23.201332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 912/1798] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:08:23 2022] POST /identity/v3/projects => generated 355 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:23.202465 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94a62b-9217-4061-b55a-b18377699288 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:23.249734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616891f-3452-4237-825e-0e3df295ff85 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:23.271504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94a62b-9217-4061-b55a-b18377699288 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:23.276031 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a49b9-15d4-44e4-a2df-852b91ec61b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:23.276308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a49b9-15d4-44e4-a2df-852b91ec61b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:23.276598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a49b9-15d4-44e4-a2df-852b91ec61b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:23.276883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a49b9-15d4-44e4-a2df-852b91ec61b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:23.324722 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804cb97-b56b-4dfb-a0b2-908db6cdfe31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:23.325115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804cb97-b56b-4dfb-a0b2-908db6cdfe31 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:23.325456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804cb97-b56b-4dfb-a0b2-908db6cdfe31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:23.325709 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804cb97-b56b-4dfb-a0b2-908db6cdfe31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:23.327652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8804cb97-b56b-4dfb-a0b2-908db6cdfe31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:23.341968 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a49b9-15d4-44e4-a2df-852b91ec61b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:23.343735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a49b9-15d4-44e4-a2df-852b91ec61b0 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:23.354349 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 913/1799] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:23.355436 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a49b9-15d4-44e4-a2df-852b91ec61b0 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:23.422643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804cb97-b56b-4dfb-a0b2-908db6cdfe31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:23.424617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804cb97-b56b-4dfb-a0b2-908db6cdfe31 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:23.432561 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a49b9-15d4-44e4-a2df-852b91ec61b0 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:23.434929 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 887/1800] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:23 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:23.447140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99005168-7691-437a-ae0e-1368e61952fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:23.447140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99005168-7691-437a-ae0e-1368e61952fd admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:23.447140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99005168-7691-437a-ae0e-1368e61952fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:23.447140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99005168-7691-437a-ae0e-1368e61952fd admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:23.447759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99005168-7691-437a-ae0e-1368e61952fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:23.447759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99005168-7691-437a-ae0e-1368e61952fd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:23.577148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 914/1801] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:08:23 2022] POST /identity/v3/users => generated 278 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:23.579180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804cb97-b56b-4dfb-a0b2-908db6cdfe31 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:23.591095 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804cb97-b56b-4dfb-a0b2-908db6cdfe31 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:23.598042 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43f913-3aac-473c-8123-423f27a16801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:23.598183 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99005168-7691-437a-ae0e-1368e61952fd admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:23.598281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43f913-3aac-473c-8123-423f27a16801 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:23.598281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43f913-3aac-473c-8123-423f27a16801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:23.598281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43f913-3aac-473c-8123-423f27a16801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:23.598281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43f913-3aac-473c-8123-423f27a16801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:23.599211 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43f913-3aac-473c-8123-423f27a16801 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:23.606766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99005168-7691-437a-ae0e-1368e61952fd admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:23.609661 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 888/1802] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:23.615315 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43f913-3aac-473c-8123-423f27a16801 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:23.616432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455f0c6-2ccd-41df-b795-295a5af380cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:23.617108 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455f0c6-2ccd-41df-b795-295a5af380cc service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:23.617396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455f0c6-2ccd-41df-b795-295a5af380cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:23.617676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455f0c6-2ccd-41df-b795-295a5af380cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:23.619677 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e455f0c6-2ccd-41df-b795-295a5af380cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:23.625917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43f913-3aac-473c-8123-423f27a16801 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:23.630823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf2a6f-e454-46bc-8df9-dfd986e18d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:23.631501 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf2a6f-e454-46bc-8df9-dfd986e18d02 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:23.631791 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf2a6f-e454-46bc-8df9-dfd986e18d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:23.632070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf2a6f-e454-46bc-8df9-dfd986e18d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:23.632833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf2a6f-e454-46bc-8df9-dfd986e18d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:23.634424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf2a6f-e454-46bc-8df9-dfd986e18d02 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:23.645430 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 889/1803] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:08:23.649718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf2a6f-e454-46bc-8df9-dfd986e18d02 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:23.659487 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf2a6f-e454-46bc-8df9-dfd986e18d02 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:23.664165 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563363ba-dc41-434c-bb41-6288cc31419a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:23.665074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563363ba-dc41-434c-bb41-6288cc31419a admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:23.665361 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563363ba-dc41-434c-bb41-6288cc31419a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:23.665749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563363ba-dc41-434c-bb41-6288cc31419a admin admin] PATH_INFO: `/v3/projects/faf21454928a4888b102f0683c19cff6/users/aed2e1102ad341f2a34144e4f9ce5800/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:23.680195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563363ba-dc41-434c-bb41-6288cc31419a admin admin] RBAC: Authorizing `identity:create_grant(project_id=faf21454928a4888b102f0683c19cff6, user_id=aed2e1102ad341f2a34144e4f9ce5800, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:23.682083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e455f0c6-2ccd-41df-b795-295a5af380cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:23.683575 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e455f0c6-2ccd-41df-b795-295a5af380cc service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:23.688756 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563363ba-dc41-434c-bb41-6288cc31419a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:23.704339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 915/1804] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:08:23.758639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 890/1805] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:23 2022] PUT /identity/v3/projects/faf21454928a4888b102f0683c19cff6/users/aed2e1102ad341f2a34144e4f9ce5800/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:08:23.769373 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523b859-6c24-40e7-b27d-1ea714d618d2 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:23.769667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523b859-6c24-40e7-b27d-1ea714d618d2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:23.769953 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523b859-6c24-40e7-b27d-1ea714d618d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:23.798461 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-1523b859-6c24-40e7-b27d-1ea714d618d2 None None] MFA Rules not processed for user `aed2e1102ad341f2a34144e4f9ce5800`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:23.799723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1523b859-6c24-40e7-b27d-1ea714d618d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:23.871353 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 916/1806] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:23 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:23.876492 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563363ba-dc41-434c-bb41-6288cc31419a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:23.943418 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563363ba-dc41-434c-bb41-6288cc31419a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:23.948086 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddac4df-136c-4520-9985-ea68e3a9160e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:23.948744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddac4df-136c-4520-9985-ea68e3a9160e admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:23.949029 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddac4df-136c-4520-9985-ea68e3a9160e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:23.949310 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddac4df-136c-4520-9985-ea68e3a9160e admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:23.950089 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddac4df-136c-4520-9985-ea68e3a9160e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:23.952050 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddac4df-136c-4520-9985-ea68e3a9160e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:24.068000 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 891/1807] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:08:23 2022] POST /identity/v3/projects => generated 355 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:24.072244 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523b859-6c24-40e7-b27d-1ea714d618d2 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:24.082964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523b859-6c24-40e7-b27d-1ea714d618d2 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:24.142238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db353f9-4bc0-4b61-b67e-cb6712323dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:24.142952 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db353f9-4bc0-4b61-b67e-cb6712323dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:24.143265 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db353f9-4bc0-4b61-b67e-cb6712323dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:24.143547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db353f9-4bc0-4b61-b67e-cb6712323dc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:24.144304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db353f9-4bc0-4b61-b67e-cb6712323dc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:24.146637 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db353f9-4bc0-4b61-b67e-cb6712323dc8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:24.281988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 917/1808] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:08:24 2022] POST /identity/v3/users => generated 284 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:24.286280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddac4df-136c-4520-9985-ea68e3a9160e admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:24.300792 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddac4df-136c-4520-9985-ea68e3a9160e admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:24.315782 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69017631-57ba-4661-a4e6-679b80258d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:24.316499 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69017631-57ba-4661-a4e6-679b80258d57 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:24.316793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69017631-57ba-4661-a4e6-679b80258d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:24.318932 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69017631-57ba-4661-a4e6-679b80258d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:24.322685 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69017631-57ba-4661-a4e6-679b80258d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:24.324337 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69017631-57ba-4661-a4e6-679b80258d57 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:24.336060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 892/1809] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:24.344215 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db353f9-4bc0-4b61-b67e-cb6712323dc8 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:24.355383 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db353f9-4bc0-4b61-b67e-cb6712323dc8 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:24.369398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019a84f-8036-45d7-9a58-50ed6088f9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:24.369398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019a84f-8036-45d7-9a58-50ed6088f9d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:24.369398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019a84f-8036-45d7-9a58-50ed6088f9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:24.369398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019a84f-8036-45d7-9a58-50ed6088f9d9 admin admin] PATH_INFO: `/v3/projects/187b9e7d610b4718bf609777d4033ff2/users/f6464d9246364819a85f0b7e3679a83a/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:24.380053 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019a84f-8036-45d7-9a58-50ed6088f9d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=187b9e7d610b4718bf609777d4033ff2, user_id=f6464d9246364819a85f0b7e3679a83a, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:24.382856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019a84f-8036-45d7-9a58-50ed6088f9d9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:24.484500 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 918/1810] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:24 2022] PUT /identity/v3/projects/187b9e7d610b4718bf609777d4033ff2/users/f6464d9246364819a85f0b7e3679a83a/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:24.488442 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69017631-57ba-4661-a4e6-679b80258d57 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:24.498904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69017631-57ba-4661-a4e6-679b80258d57 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:24.555503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff13c7-8cb4-49cb-91a8-cbeb5724da4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:24.556192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff13c7-8cb4-49cb-91a8-cbeb5724da4b admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:24.556484 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff13c7-8cb4-49cb-91a8-cbeb5724da4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:24.556768 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff13c7-8cb4-49cb-91a8-cbeb5724da4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:24.557497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff13c7-8cb4-49cb-91a8-cbeb5724da4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:24.559202 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff13c7-8cb4-49cb-91a8-cbeb5724da4b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:24.569168 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 893/1811] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:24.573385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019a84f-8036-45d7-9a58-50ed6088f9d9 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:24.584090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019a84f-8036-45d7-9a58-50ed6088f9d9 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:24.588635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1ba81-2465-4bd0-b803-3c6aab75cc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:24.589511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1ba81-2465-4bd0-b803-3c6aab75cc67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:24.589794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1ba81-2465-4bd0-b803-3c6aab75cc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:24.590071 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1ba81-2465-4bd0-b803-3c6aab75cc67 admin admin] PATH_INFO: `/v3/projects/187b9e7d610b4718bf609777d4033ff2/users/f6464d9246364819a85f0b7e3679a83a/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:24.592565 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1ba81-2465-4bd0-b803-3c6aab75cc67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=187b9e7d610b4718bf609777d4033ff2, user_id=f6464d9246364819a85f0b7e3679a83a, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:24.595221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1ba81-2465-4bd0-b803-3c6aab75cc67 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:24.607919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 919/1812] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:24 2022] PUT /identity/v3/projects/187b9e7d610b4718bf609777d4033ff2/users/f6464d9246364819a85f0b7e3679a83a/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:24.614330 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:24.618920 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525e94d-1045-4883-8163-5f9594337e6c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:24.619386 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525e94d-1045-4883-8163-5f9594337e6c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:24.619680 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525e94d-1045-4883-8163-5f9594337e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:24.648506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0525e94d-1045-4883-8163-5f9594337e6c None None] MFA Rules not processed for user `f6464d9246364819a85f0b7e3679a83a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:24.649956 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0525e94d-1045-4883-8163-5f9594337e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:24.731050 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 894/1813] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:24 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:24.743046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1ba81-2465-4bd0-b803-3c6aab75cc67 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:24.848323 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1ba81-2465-4bd0-b803-3c6aab75cc67 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:24.848323 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca042d-907f-4f8c-b4de-3458d7e3769d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:24.848323 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca042d-907f-4f8c-b4de-3458d7e3769d service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:24.848323 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca042d-907f-4f8c-b4de-3458d7e3769d service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:24.848772 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca042d-907f-4f8c-b4de-3458d7e3769d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:24.850969 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca042d-907f-4f8c-b4de-3458d7e3769d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:24.852455 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca042d-907f-4f8c-b4de-3458d7e3769d service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:24.868383 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:24.869196 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 920/1814] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:24 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:25.214115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525e94d-1045-4883-8163-5f9594337e6c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:25.228499 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525e94d-1045-4883-8163-5f9594337e6c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:25.228647 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3254fd-964e-4eae-aed9-30e915c1cb3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:25.229692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3254fd-964e-4eae-aed9-30e915c1cb3b service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:25.229976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3254fd-964e-4eae-aed9-30e915c1cb3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:25.230257 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3254fd-964e-4eae-aed9-30e915c1cb3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:25.301296 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3254fd-964e-4eae-aed9-30e915c1cb3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:25.302517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3254fd-964e-4eae-aed9-30e915c1cb3b service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:25.314643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 895/1815] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:25 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:08:25.609040 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:25.666709 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:26.013713 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca042d-907f-4f8c-b4de-3458d7e3769d service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:26.018198 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca042d-907f-4f8c-b4de-3458d7e3769d service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:26.022217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9ba92-0058-4043-9b52-591eb9f5b846 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:26.022917 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9ba92-0058-4043-9b52-591eb9f5b846 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:26.023232 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9ba92-0058-4043-9b52-591eb9f5b846 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:26.023511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9ba92-0058-4043-9b52-591eb9f5b846 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:26.025265 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e3c9ba92-0058-4043-9b52-591eb9f5b846 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:26.127930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9ba92-0058-4043-9b52-591eb9f5b846 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:26.130156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9ba92-0058-4043-9b52-591eb9f5b846 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:26.150316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 921/1816] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:26.300359 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:26.363583 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:26.900678 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:27.552179 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:27.803451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:30.366774 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3254fd-964e-4eae-aed9-30e915c1cb3b service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:30.368356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9e3254fd-964e-4eae-aed9-30e915c1cb3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:30.445155 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3254fd-964e-4eae-aed9-30e915c1cb3b service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:30.449846 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d31f5-3ff4-4fa5-974f-622ee4e674f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:30.450728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d31f5-3ff4-4fa5-974f-622ee4e674f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:30.451009 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d31f5-3ff4-4fa5-974f-622ee4e674f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:30.451316 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d31f5-3ff4-4fa5-974f-622ee4e674f6 admin admin] PATH_INFO: `/v3/users/392898e8ef8446e9938e205f5cbd92d5` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:30.452523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d31f5-3ff4-4fa5-974f-622ee4e674f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=392898e8ef8446e9938e205f5cbd92d5)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:30.454497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d31f5-3ff4-4fa5-974f-622ee4e674f6 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:30.657553 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9c4d31f5-3ff4-4fa5-974f-622ee4e674f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '392898e8ef8446e9938e205f5cbd92d5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:30.664426 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9c4d31f5-3ff4-4fa5-974f-622ee4e674f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '392898e8ef8446e9938e205f5cbd92d5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:30.665108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9c4d31f5-3ff4-4fa5-974f-622ee4e674f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '392898e8ef8446e9938e205f5cbd92d5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:30.665714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9c4d31f5-3ff4-4fa5-974f-622ee4e674f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '392898e8ef8446e9938e205f5cbd92d5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:30.680044 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9c4d31f5-3ff4-4fa5-974f-622ee4e674f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '392898e8ef8446e9938e205f5cbd92d5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:30.682109 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:30.763250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 896/1817] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:08:30 2022] DELETE /identity/v3/users/392898e8ef8446e9938e205f5cbd92d5 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:30.768046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c10b8-d155-49da-9dbe-bef0f18c3619 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:30.768564 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c10b8-d155-49da-9dbe-bef0f18c3619 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:30.768661 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c10b8-d155-49da-9dbe-bef0f18c3619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:30.786631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-f16c10b8-d155-49da-9dbe-bef0f18c3619 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:30.787874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f16c10b8-d155-49da-9dbe-bef0f18c3619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:30.850822 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 922/1818] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:30.866914 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d31f5-3ff4-4fa5-974f-622ee4e674f6 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:30.866914 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9c4d31f5-3ff4-4fa5-974f-622ee4e674f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:30.923988 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d31f5-3ff4-4fa5-974f-622ee4e674f6 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:30.927571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b5e81-2450-47d2-bd9b-cac402da99a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:30.928220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b5e81-2450-47d2-bd9b-cac402da99a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:30.928507 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b5e81-2450-47d2-bd9b-cac402da99a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:30.928791 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b5e81-2450-47d2-bd9b-cac402da99a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:30.938435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b5e81-2450-47d2-bd9b-cac402da99a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:30.939789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b5e81-2450-47d2-bd9b-cac402da99a3 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:30.951179 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 897/1819] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:31.399294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c10b8-d155-49da-9dbe-bef0f18c3619 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:31.400850 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f16c10b8-d155-49da-9dbe-bef0f18c3619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:31.417532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c10b8-d155-49da-9dbe-bef0f18c3619 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:31.424881 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808d7cd-c4b9-42ba-a823-ad3648d2ceaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:31.427472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808d7cd-c4b9-42ba-a823-ad3648d2ceaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:31.427782 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808d7cd-c4b9-42ba-a823-ad3648d2ceaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:31.428069 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808d7cd-c4b9-42ba-a823-ad3648d2ceaa admin admin] PATH_INFO: `/v3/projects/50e1644b846a4f5aa76e84fac98cd25e` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:31.429351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4808d7cd-c4b9-42ba-a823-ad3648d2ceaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=50e1644b846a4f5aa76e84fac98cd25e)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:31.431717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4808d7cd-c4b9-42ba-a823-ad3648d2ceaa admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:31.536861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:31.539519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4808d7cd-c4b9-42ba-a823-ad3648d2ceaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50e1644b846a4f5aa76e84fac98cd25e', 'request_id': 'req-4808d7cd-c4b9-42ba-a823-ad3648d2ceaa'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:31.608414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4808d7cd-c4b9-42ba-a823-ad3648d2ceaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50e1644b846a4f5aa76e84fac98cd25e', 'request_id': 'req-4808d7cd-c4b9-42ba-a823-ad3648d2ceaa'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:31.614402 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4808d7cd-c4b9-42ba-a823-ad3648d2ceaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50e1644b846a4f5aa76e84fac98cd25e', 'request_id': 'req-4808d7cd-c4b9-42ba-a823-ad3648d2ceaa'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:31.628624 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4808d7cd-c4b9-42ba-a823-ad3648d2ceaa admin admin] The token cache is being invalidate because project 50e1644b846a4f5aa76e84fac98cd25e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:08:31.628919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4808d7cd-c4b9-42ba-a823-ad3648d2ceaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:31.629590 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4808d7cd-c4b9-42ba-a823-ad3648d2ceaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:31.631159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 923/1820] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:08:31 2022] DELETE /identity/v3/projects/50e1644b846a4f5aa76e84fac98cd25e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:31.635083 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b5e81-2450-47d2-bd9b-cac402da99a3 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:31.645036 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b5e81-2450-47d2-bd9b-cac402da99a3 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:31.706548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb4f2a-4bd6-4bb0-b370-7b191dbe7ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:31.707467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb4f2a-4bd6-4bb0-b370-7b191dbe7ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:31.707756 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb4f2a-4bd6-4bb0-b370-7b191dbe7ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:31.708036 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb4f2a-4bd6-4bb0-b370-7b191dbe7ead admin admin] PATH_INFO: `/v3/users/7aa5fbf6170d40ab85f394a213e66c85` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:31.709371 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb4f2a-4bd6-4bb0-b370-7b191dbe7ead admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aa5fbf6170d40ab85f394a213e66c85)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:31.711502 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb4f2a-4bd6-4bb0-b370-7b191dbe7ead admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:31.894008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9acb4f2a-4bd6-4bb0-b370-7b191dbe7ead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aa5fbf6170d40ab85f394a213e66c85'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:31.898923 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9acb4f2a-4bd6-4bb0-b370-7b191dbe7ead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aa5fbf6170d40ab85f394a213e66c85'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:31.899672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9acb4f2a-4bd6-4bb0-b370-7b191dbe7ead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aa5fbf6170d40ab85f394a213e66c85'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:31.900298 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9acb4f2a-4bd6-4bb0-b370-7b191dbe7ead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aa5fbf6170d40ab85f394a213e66c85'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:31.912659 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9acb4f2a-4bd6-4bb0-b370-7b191dbe7ead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aa5fbf6170d40ab85f394a213e66c85'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:32.003024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 898/1821] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:08:31 2022] DELETE /identity/v3/users/7aa5fbf6170d40ab85f394a213e66c85 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:32.447338 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808d7cd-c4b9-42ba-a823-ad3648d2ceaa admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:32.448802 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4808d7cd-c4b9-42ba-a823-ad3648d2ceaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:32.462951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808d7cd-c4b9-42ba-a823-ad3648d2ceaa admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:32.526778 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1fa26d-398f-4343-a51f-078589cdab85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:32.528310 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fa26d-398f-4343-a51f-078589cdab85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:32.528598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fa26d-398f-4343-a51f-078589cdab85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:32.528901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fa26d-398f-4343-a51f-078589cdab85 admin admin] PATH_INFO: `/v3/projects/4f10ac6940af45859e2606ba086e7f7d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:32.530532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1fa26d-398f-4343-a51f-078589cdab85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f10ac6940af45859e2606ba086e7f7d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:32.532545 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1fa26d-398f-4343-a51f-078589cdab85 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:32.684745 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6e1fa26d-398f-4343-a51f-078589cdab85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f10ac6940af45859e2606ba086e7f7d', 'request_id': 'req-6e1fa26d-398f-4343-a51f-078589cdab85'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:32.780031 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6e1fa26d-398f-4343-a51f-078589cdab85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f10ac6940af45859e2606ba086e7f7d', 'request_id': 'req-6e1fa26d-398f-4343-a51f-078589cdab85'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:32.786268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6e1fa26d-398f-4343-a51f-078589cdab85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f10ac6940af45859e2606ba086e7f7d', 'request_id': 'req-6e1fa26d-398f-4343-a51f-078589cdab85'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:32.800576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6e1fa26d-398f-4343-a51f-078589cdab85 admin admin] The token cache is being invalidate because project 4f10ac6940af45859e2606ba086e7f7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:08:32.800886 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6e1fa26d-398f-4343-a51f-078589cdab85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:32.801553 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6e1fa26d-398f-4343-a51f-078589cdab85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:32.803121 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 924/1822] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:08:32 2022] DELETE /identity/v3/projects/4f10ac6940af45859e2606ba086e7f7d => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:32.808454 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d74fa-07d3-4a7e-ba7f-d4f963da342c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:32.809462 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d74fa-07d3-4a7e-ba7f-d4f963da342c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:32.809577 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d74fa-07d3-4a7e-ba7f-d4f963da342c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:32.831622 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-524d74fa-07d3-4a7e-ba7f-d4f963da342c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:32.832833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-524d74fa-07d3-4a7e-ba7f-d4f963da342c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:32.841760 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:32.895489 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 899/1823] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:32.904219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f21e9d-a8cc-4c15-aeb3-1aba39af0a6b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:32.904534 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f21e9d-a8cc-4c15-aeb3-1aba39af0a6b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:32.904843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f21e9d-a8cc-4c15-aeb3-1aba39af0a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:32.925398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e2f21e9d-a8cc-4c15-aeb3-1aba39af0a6b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:32.926552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e2f21e9d-a8cc-4c15-aeb3-1aba39af0a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:32.935352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 925/1824] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:32.939666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d74fa-07d3-4a7e-ba7f-d4f963da342c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:32.949343 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d74fa-07d3-4a7e-ba7f-d4f963da342c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:32.954564 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63273f5d-71b7-47d5-a896-81e3b545491c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:32.955248 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63273f5d-71b7-47d5-a896-81e3b545491c admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:32.955537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63273f5d-71b7-47d5-a896-81e3b545491c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:32.957367 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63273f5d-71b7-47d5-a896-81e3b545491c admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:32.957367 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63273f5d-71b7-47d5-a896-81e3b545491c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:08:32.957367 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63273f5d-71b7-47d5-a896-81e3b545491c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:32.958449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63273f5d-71b7-47d5-a896-81e3b545491c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:32.970067 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 900/1825] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:08:32.974397 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f21e9d-a8cc-4c15-aeb3-1aba39af0a6b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:32.984192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f21e9d-a8cc-4c15-aeb3-1aba39af0a6b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:32.989188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d62f7-70db-4efb-8251-cd444b45c163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:32.989921 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d62f7-70db-4efb-8251-cd444b45c163 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:32.990379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d62f7-70db-4efb-8251-cd444b45c163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:32.990663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d62f7-70db-4efb-8251-cd444b45c163 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:32.991425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d62f7-70db-4efb-8251-cd444b45c163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:32.993415 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d62f7-70db-4efb-8251-cd444b45c163 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:33.075381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 926/1826] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:08:32 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:33.079646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63273f5d-71b7-47d5-a896-81e3b545491c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:33.090892 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63273f5d-71b7-47d5-a896-81e3b545491c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:33.153051 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1046b-87f2-4193-b17d-c8a85c1ce1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:33.154352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1046b-87f2-4193-b17d-c8a85c1ce1d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:33.154643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1046b-87f2-4193-b17d-c8a85c1ce1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:33.154927 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1046b-87f2-4193-b17d-c8a85c1ce1d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:33.155677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1046b-87f2-4193-b17d-c8a85c1ce1d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:33.157611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1046b-87f2-4193-b17d-c8a85c1ce1d4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:33.269728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 901/1827] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:08:33 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:33.273861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d62f7-70db-4efb-8251-cd444b45c163 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:33.294322 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d62f7-70db-4efb-8251-cd444b45c163 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:33.294322 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb6d42-5e5c-43bf-8080-4a7e6c7ee911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:33.294322 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb6d42-5e5c-43bf-8080-4a7e6c7ee911 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:33.294322 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb6d42-5e5c-43bf-8080-4a7e6c7ee911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:33.295035 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb6d42-5e5c-43bf-8080-4a7e6c7ee911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:33.295035 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb6d42-5e5c-43bf-8080-4a7e6c7ee911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:33.295932 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb6d42-5e5c-43bf-8080-4a7e6c7ee911 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:33.319490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 927/1828] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:33.322388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1046b-87f2-4193-b17d-c8a85c1ce1d4 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:33.334033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1046b-87f2-4193-b17d-c8a85c1ce1d4 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:33.339869 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd265827-81e3-400e-96f8-fa2952918d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:33.340929 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd265827-81e3-400e-96f8-fa2952918d03 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:33.341060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd265827-81e3-400e-96f8-fa2952918d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:33.341424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd265827-81e3-400e-96f8-fa2952918d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:33.342248 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd265827-81e3-400e-96f8-fa2952918d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:33.346874 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd265827-81e3-400e-96f8-fa2952918d03 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:33.357239 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 902/1829] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:33.361363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb6d42-5e5c-43bf-8080-4a7e6c7ee911 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:33.373248 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb6d42-5e5c-43bf-8080-4a7e6c7ee911 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:33.379929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec4b1a-1e08-4ffc-9df0-a9b5a224625c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:33.381019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec4b1a-1e08-4ffc-9df0-a9b5a224625c admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:33.381378 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec4b1a-1e08-4ffc-9df0-a9b5a224625c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:33.381778 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec4b1a-1e08-4ffc-9df0-a9b5a224625c admin admin] PATH_INFO: `/v3/projects/7016950331214535a290476500499e48/users/df9ef565283b477dae0cc83d13a37765/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:33.402986 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec4b1a-1e08-4ffc-9df0-a9b5a224625c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7016950331214535a290476500499e48, user_id=df9ef565283b477dae0cc83d13a37765, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:33.406126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec4b1a-1e08-4ffc-9df0-a9b5a224625c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:33.411273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:33.563912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 928/1830] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:33 2022] PUT /identity/v3/projects/7016950331214535a290476500499e48/users/df9ef565283b477dae0cc83d13a37765/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:33.574267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01953f20-120b-47c0-9705-1d622cae7faa None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:33.574753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01953f20-120b-47c0-9705-1d622cae7faa None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:33.575870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01953f20-120b-47c0-9705-1d622cae7faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:33.632808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-01953f20-120b-47c0-9705-1d622cae7faa None None] MFA Rules not processed for user `df9ef565283b477dae0cc83d13a37765`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:33.635885 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-01953f20-120b-47c0-9705-1d622cae7faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:33.733319 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 903/1831] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:33.737931 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec4b1a-1e08-4ffc-9df0-a9b5a224625c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:33.832660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec4b1a-1e08-4ffc-9df0-a9b5a224625c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:33.838712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a6005-7384-4eca-9be3-07f0a4ebbcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:33.842406 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a6005-7384-4eca-9be3-07f0a4ebbcd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:33.842717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a6005-7384-4eca-9be3-07f0a4ebbcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:33.842999 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a6005-7384-4eca-9be3-07f0a4ebbcd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:33.843834 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a6005-7384-4eca-9be3-07f0a4ebbcd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:33.845768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a6005-7384-4eca-9be3-07f0a4ebbcd0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:34.016819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 929/1832] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:08:33 2022] POST /identity/v3/projects => generated 385 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:34.017953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01953f20-120b-47c0-9705-1d622cae7faa None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:34.029713 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01953f20-120b-47c0-9705-1d622cae7faa None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:34.034322 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:34.090051 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e278c79-ed56-4bc6-a251-420ec42148a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:34.090888 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e278c79-ed56-4bc6-a251-420ec42148a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:34.091245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e278c79-ed56-4bc6-a251-420ec42148a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:34.091535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e278c79-ed56-4bc6-a251-420ec42148a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:34.092256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e278c79-ed56-4bc6-a251-420ec42148a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:34.094216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e278c79-ed56-4bc6-a251-420ec42148a2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:34.179490 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 904/1833] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:08:34 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:08:34.183738 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a6005-7384-4eca-9be3-07f0a4ebbcd0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:34.194821 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a6005-7384-4eca-9be3-07f0a4ebbcd0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:34.199507 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b8486-a914-4e0b-bf4f-f2dd5c604b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:34.200810 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b8486-a914-4e0b-bf4f-f2dd5c604b0e admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:34.201197 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b8486-a914-4e0b-bf4f-f2dd5c604b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:34.201373 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b8486-a914-4e0b-bf4f-f2dd5c604b0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:34.202110 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b8486-a914-4e0b-bf4f-f2dd5c604b0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:34.203775 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b8486-a914-4e0b-bf4f-f2dd5c604b0e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:34.214777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 930/1834] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:34.219284 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e278c79-ed56-4bc6-a251-420ec42148a2 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:34.229640 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e278c79-ed56-4bc6-a251-420ec42148a2 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:34.234602 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd2b30-887e-40e6-9c83-925e1dc12deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:34.235437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd2b30-887e-40e6-9c83-925e1dc12deb admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:34.235726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd2b30-887e-40e6-9c83-925e1dc12deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:34.236004 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd2b30-887e-40e6-9c83-925e1dc12deb admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:34.236729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd2b30-887e-40e6-9c83-925e1dc12deb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:34.238328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd2b30-887e-40e6-9c83-925e1dc12deb admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:34.249133 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 905/1835] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:34.255135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b8486-a914-4e0b-bf4f-f2dd5c604b0e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:34.265892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b8486-a914-4e0b-bf4f-f2dd5c604b0e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:34.270365 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae1e2e-8232-4f69-9575-9df9b4ca13e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:34.271299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae1e2e-8232-4f69-9575-9df9b4ca13e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:34.271569 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae1e2e-8232-4f69-9575-9df9b4ca13e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:34.271875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae1e2e-8232-4f69-9575-9df9b4ca13e7 admin admin] PATH_INFO: `/v3/projects/4aac0577bf2f4a829d43f3e3d3c8d29b/users/7d66ff536bdd4e53b84733439ce50d30/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:34.288561 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae1e2e-8232-4f69-9575-9df9b4ca13e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aac0577bf2f4a829d43f3e3d3c8d29b, user_id=7d66ff536bdd4e53b84733439ce50d30, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:34.291517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae1e2e-8232-4f69-9575-9df9b4ca13e7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:34.413488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 931/1836] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:34 2022] PUT /identity/v3/projects/4aac0577bf2f4a829d43f3e3d3c8d29b/users/7d66ff536bdd4e53b84733439ce50d30/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:08:34.425340 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797439a-be72-4a6e-a0fc-308cd3c83dc8 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:34.425759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797439a-be72-4a6e-a0fc-308cd3c83dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:34.426146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797439a-be72-4a6e-a0fc-308cd3c83dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:34.462311 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0797439a-be72-4a6e-a0fc-308cd3c83dc8 None None] MFA Rules not processed for user `7d66ff536bdd4e53b84733439ce50d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:34.462311 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0797439a-be72-4a6e-a0fc-308cd3c83dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:34.534112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 906/1837] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:34.535277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae1e2e-8232-4f69-9575-9df9b4ca13e7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:34.598289 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae1e2e-8232-4f69-9575-9df9b4ca13e7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:34.603277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdaadf8-92ce-41b9-8781-920fe6e70045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:34.604060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdaadf8-92ce-41b9-8781-920fe6e70045 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:34.604351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdaadf8-92ce-41b9-8781-920fe6e70045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:34.604657 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdaadf8-92ce-41b9-8781-920fe6e70045 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:34.605403 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdaadf8-92ce-41b9-8781-920fe6e70045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:34.607485 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdaadf8-92ce-41b9-8781-920fe6e70045 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:34.756017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 932/1838] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:08:34 2022] POST /identity/v3/projects => generated 387 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:08:34.760226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797439a-be72-4a6e-a0fc-308cd3c83dc8 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:34.772940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797439a-be72-4a6e-a0fc-308cd3c83dc8 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:34.800867 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f533748-cb3b-4e61-8ecc-c4adf949c387 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:34.801232 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f533748-cb3b-4e61-8ecc-c4adf949c387 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:34.809496 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f533748-cb3b-4e61-8ecc-c4adf949c387 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:34.809496 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 933/1839] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 14 19:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 19:08:34.831812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be989f-20b4-45ec-bc92-56ec182311b8 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:34.832192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be989f-20b4-45ec-bc92-56ec182311b8 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:34.832540 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be989f-20b4-45ec-bc92-56ec182311b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:34.873007 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-41be989f-20b4-45ec-bc92-56ec182311b8 None None] MFA Rules not processed for user `8d27d6df5cb64697adebd961dfbec324`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:34.874439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-41be989f-20b4-45ec-bc92-56ec182311b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:34.893287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb604ca-5322-4022-a7cc-8297d863925b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:34.895931 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb604ca-5322-4022-a7cc-8297d863925b admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:34.898502 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb604ca-5322-4022-a7cc-8297d863925b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:34.898946 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb604ca-5322-4022-a7cc-8297d863925b admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:34.900002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb604ca-5322-4022-a7cc-8297d863925b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:34.903923 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb604ca-5322-4022-a7cc-8297d863925b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:34.916806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:34.965904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 934/1840] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 14 19:08:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:08:35.031724 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 907/1841] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:08:34 2022] POST /identity/v3/users => generated 300 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:08:35.033565 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be989f-20b4-45ec-bc92-56ec182311b8 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:35.048207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be989f-20b4-45ec-bc92-56ec182311b8 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:35.053171 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1d563-5cee-4d30-9b1d-08c7e3657bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:35.053913 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1d563-5cee-4d30-9b1d-08c7e3657bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:35.054198 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1d563-5cee-4d30-9b1d-08c7e3657bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:35.054818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1d563-5cee-4d30-9b1d-08c7e3657bb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:35.055229 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1d563-5cee-4d30-9b1d-08c7e3657bb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:35.056953 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1d563-5cee-4d30-9b1d-08c7e3657bb7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:35.072343 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 935/1842] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:35.077178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb604ca-5322-4022-a7cc-8297d863925b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:35.089055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb604ca-5322-4022-a7cc-8297d863925b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:35.095437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcecf1e-7278-4dbc-b271-6f3d21603aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:35.096581 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcecf1e-7278-4dbc-b271-6f3d21603aa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:35.096945 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcecf1e-7278-4dbc-b271-6f3d21603aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:35.097308 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcecf1e-7278-4dbc-b271-6f3d21603aa0 admin admin] PATH_INFO: `/v3/projects/0ca976e996c549d5a8a216c4c906e18b/users/81210649f5ac4a8da32c9fad677f663d/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:35.101360 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:35.124074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcecf1e-7278-4dbc-b271-6f3d21603aa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ca976e996c549d5a8a216c4c906e18b, user_id=81210649f5ac4a8da32c9fad677f663d, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:35.130597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcecf1e-7278-4dbc-b271-6f3d21603aa0 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:35.251868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 908/1843] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:35 2022] PUT /identity/v3/projects/0ca976e996c549d5a8a216c4c906e18b/users/81210649f5ac4a8da32c9fad677f663d/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:35.258696 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1d563-5cee-4d30-9b1d-08c7e3657bb7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:35.276308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1d563-5cee-4d30-9b1d-08c7e3657bb7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:35.285089 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcecf1e-7278-4dbc-b271-6f3d21603aa0 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:35.292825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6bcecf1e-7278-4dbc-b271-6f3d21603aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:35.348051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63994df6-d5d9-4384-b16a-d2cd74a38c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:35.348834 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63994df6-d5d9-4384-b16a-d2cd74a38c1b admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:35.349128 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63994df6-d5d9-4384-b16a-d2cd74a38c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:35.349435 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63994df6-d5d9-4384-b16a-d2cd74a38c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:35.350173 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63994df6-d5d9-4384-b16a-d2cd74a38c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:35.351836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63994df6-d5d9-4384-b16a-d2cd74a38c1b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:35.363942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 936/1844] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:08:35.368134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63994df6-d5d9-4384-b16a-d2cd74a38c1b admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:35.381797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcecf1e-7278-4dbc-b271-6f3d21603aa0 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:35.386622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63994df6-d5d9-4384-b16a-d2cd74a38c1b admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:35.393896 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f1488-a8d3-4bb3-952b-2ee18523b3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:35.394888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f1488-a8d3-4bb3-952b-2ee18523b3f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:35.395304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f1488-a8d3-4bb3-952b-2ee18523b3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:35.395686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f1488-a8d3-4bb3-952b-2ee18523b3f7 admin admin] PATH_INFO: `/v3/projects/0ca976e996c549d5a8a216c4c906e18b/users/81210649f5ac4a8da32c9fad677f663d/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:35.398213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f1488-a8d3-4bb3-952b-2ee18523b3f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ca976e996c549d5a8a216c4c906e18b, user_id=81210649f5ac4a8da32c9fad677f663d, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:35.400790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3b22e-5d4b-495e-b1ff-599418b73c98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:35.402023 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3b22e-5d4b-495e-b1ff-599418b73c98 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:35.402382 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f1488-a8d3-4bb3-952b-2ee18523b3f7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:35.402625 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3b22e-5d4b-495e-b1ff-599418b73c98 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:35.403564 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3b22e-5d4b-495e-b1ff-599418b73c98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:35.417360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 937/1845] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:08:35 2022] PUT /identity/v3/projects/0ca976e996c549d5a8a216c4c906e18b/users/81210649f5ac4a8da32c9fad677f663d/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:35.420491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3b22e-5d4b-495e-b1ff-599418b73c98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:35.422166 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3b22e-5d4b-495e-b1ff-599418b73c98 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:35.428573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608fb6b-bfd5-465f-b5d4-4eb3d39f30d3 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:35.428871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608fb6b-bfd5-465f-b5d4-4eb3d39f30d3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:35.429155 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608fb6b-bfd5-465f-b5d4-4eb3d39f30d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:35.436583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 909/1846] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:35.478446 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-4608fb6b-bfd5-465f-b5d4-4eb3d39f30d3 None None] MFA Rules not processed for user `81210649f5ac4a8da32c9fad677f663d`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:35.479726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4608fb6b-bfd5-465f-b5d4-4eb3d39f30d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:35.561714 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 938/1847] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:35 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:35.573620 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3b22e-5d4b-495e-b1ff-599418b73c98 service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:35.575086 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ccd3b22e-5d4b-495e-b1ff-599418b73c98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:35.635995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3b22e-5d4b-495e-b1ff-599418b73c98 service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:35.687083 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bb6d8-d383-40b2-870d-0787b55ccf09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:35.687698 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bb6d8-d383-40b2-870d-0787b55ccf09 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:35.687988 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bb6d8-d383-40b2-870d-0787b55ccf09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:35.688271 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bb6d8-d383-40b2-870d-0787b55ccf09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:35.697674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bb6d8-d383-40b2-870d-0787b55ccf09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:35.699210 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bb6d8-d383-40b2-870d-0787b55ccf09 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:35.710349 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 910/1848] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:36.941129 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608fb6b-bfd5-465f-b5d4-4eb3d39f30d3 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:36.943215 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4608fb6b-bfd5-465f-b5d4-4eb3d39f30d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:36.956188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608fb6b-bfd5-465f-b5d4-4eb3d39f30d3 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:36.959951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d6c95-7ec5-46df-b79b-6e4a10da9913 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:36.960763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d6c95-7ec5-46df-b79b-6e4a10da9913 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:36.961076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d6c95-7ec5-46df-b79b-6e4a10da9913 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:36.961379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d6c95-7ec5-46df-b79b-6e4a10da9913 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:36.985427 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:37.030268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d6c95-7ec5-46df-b79b-6e4a10da9913 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:37.031777 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d6c95-7ec5-46df-b79b-6e4a10da9913 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:37.042300 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 939/1849] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:45.704754 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=87,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:47.054258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=19 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:47.168268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38db33-68e9-458a-8160-1a3f4dff84cd None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:47.169797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38db33-68e9-458a-8160-1a3f4dff84cd None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:47.170283 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38db33-68e9-458a-8160-1a3f4dff84cd None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:47.172454 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 911/1850] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 14 19:08:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:08:47.178653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbfab0-893c-4c82-bc8f-2b28e9dfc177 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:47.178950 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbfab0-893c-4c82-bc8f-2b28e9dfc177 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:47.179264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbfab0-893c-4c82-bc8f-2b28e9dfc177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:47.208166 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-04bbfab0-893c-4c82-bc8f-2b28e9dfc177 None None] MFA Rules not processed for user `8d27d6df5cb64697adebd961dfbec324`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:47.211240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-04bbfab0-893c-4c82-bc8f-2b28e9dfc177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:47.333410 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 940/1851] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 14 19:08:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:47.349633 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38db33-68e9-458a-8160-1a3f4dff84cd None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:47.352619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ac38db33-68e9-458a-8160-1a3f4dff84cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:47.373559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38db33-68e9-458a-8160-1a3f4dff84cd None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:47.380544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b55ecd-3f49-4ce4-9137-6790a567cfa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:47.381479 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b55ecd-3f49-4ce4-9137-6790a567cfa1 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:47.382290 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b55ecd-3f49-4ce4-9137-6790a567cfa1 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:47.382830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b55ecd-3f49-4ce4-9137-6790a567cfa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:47.395369 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b55ecd-3f49-4ce4-9137-6790a567cfa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:47.398209 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b55ecd-3f49-4ce4-9137-6790a567cfa1 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:47.413649 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 912/1852] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:49.230318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbfab0-893c-4c82-bc8f-2b28e9dfc177 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:49.242010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbfab0-893c-4c82-bc8f-2b28e9dfc177 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:49.246713 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd5abf-81af-4bc8-aeae-1ea539541280 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:49.247400 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd5abf-81af-4bc8-aeae-1ea539541280 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:49.247687 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd5abf-81af-4bc8-aeae-1ea539541280 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:49.247966 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd5abf-81af-4bc8-aeae-1ea539541280 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:49.308773 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd5abf-81af-4bc8-aeae-1ea539541280 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:49.310334 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd5abf-81af-4bc8-aeae-1ea539541280 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:49.319652 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 941/1853] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:56.818443 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b55ecd-3f49-4ce4-9137-6790a567cfa1 service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:56.933424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b55ecd-3f49-4ce4-9137-6790a567cfa1 service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:56.941077 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8a412-cb9b-480f-864a-76198ce19464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:56.942053 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8a412-cb9b-480f-864a-76198ce19464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:56.942326 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8a412-cb9b-480f-864a-76198ce19464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:56.942636 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8a412-cb9b-480f-864a-76198ce19464 admin admin] PATH_INFO: `/v3/users/df9ef565283b477dae0cc83d13a37765` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:56.943860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8a412-cb9b-480f-864a-76198ce19464 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df9ef565283b477dae0cc83d13a37765)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:56.945884 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8a412-cb9b-480f-864a-76198ce19464 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:57.285577 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-1ad8a412-cb9b-480f-864a-76198ce19464 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df9ef565283b477dae0cc83d13a37765'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:57.314347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-1ad8a412-cb9b-480f-864a-76198ce19464 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df9ef565283b477dae0cc83d13a37765'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:57.315976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-1ad8a412-cb9b-480f-864a-76198ce19464 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df9ef565283b477dae0cc83d13a37765'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:57.316770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-1ad8a412-cb9b-480f-864a-76198ce19464 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df9ef565283b477dae0cc83d13a37765'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:57.354410 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-1ad8a412-cb9b-480f-864a-76198ce19464 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df9ef565283b477dae0cc83d13a37765'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:57.478817 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 913/1854] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:08:56 2022] DELETE /identity/v3/users/df9ef565283b477dae0cc83d13a37765 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:57.485822 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fa467-7dac-4f43-ad4b-24015c0ed7cc None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:57.486455 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fa467-7dac-4f43-ad4b-24015c0ed7cc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:57.486455 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fa467-7dac-4f43-ad4b-24015c0ed7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:57.532338 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e16fa467-7dac-4f43-ad4b-24015c0ed7cc None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:08:57.533642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e16fa467-7dac-4f43-ad4b-24015c0ed7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:57.663292 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 942/1855] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:08:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:08:57.677350 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8a412-cb9b-480f-864a-76198ce19464 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:57.682285 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1ad8a412-cb9b-480f-864a-76198ce19464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:57.766032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8a412-cb9b-480f-864a-76198ce19464 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:57.769786 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dd775-a54c-4afd-960b-3685db6de7e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:57.770452 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dd775-a54c-4afd-960b-3685db6de7e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:57.770773 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dd775-a54c-4afd-960b-3685db6de7e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:57.771057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dd775-a54c-4afd-960b-3685db6de7e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:57.781178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7dd775-a54c-4afd-960b-3685db6de7e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:57.782625 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7dd775-a54c-4afd-960b-3685db6de7e4 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:57.792912 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 914/1856] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:08:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:08:58.565025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fa467-7dac-4f43-ad4b-24015c0ed7cc None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:58.566784 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e16fa467-7dac-4f43-ad4b-24015c0ed7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:58.588806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fa467-7dac-4f43-ad4b-24015c0ed7cc None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:58.595439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2b70f-dc09-41ea-8f12-eeac74786df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:58.596342 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2b70f-dc09-41ea-8f12-eeac74786df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:58.596950 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2b70f-dc09-41ea-8f12-eeac74786df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:58.597380 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2b70f-dc09-41ea-8f12-eeac74786df2 admin admin] PATH_INFO: `/v3/projects/7016950331214535a290476500499e48` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:58.598879 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2b70f-dc09-41ea-8f12-eeac74786df2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7016950331214535a290476500499e48)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:58.601059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2b70f-dc09-41ea-8f12-eeac74786df2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:58.741689 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:58.761798 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c1d2b70f-dc09-41ea-8f12-eeac74786df2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7016950331214535a290476500499e48', 'request_id': 'req-c1d2b70f-dc09-41ea-8f12-eeac74786df2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:58.865228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c1d2b70f-dc09-41ea-8f12-eeac74786df2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7016950331214535a290476500499e48', 'request_id': 'req-c1d2b70f-dc09-41ea-8f12-eeac74786df2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:58.873564 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c1d2b70f-dc09-41ea-8f12-eeac74786df2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7016950331214535a290476500499e48', 'request_id': 'req-c1d2b70f-dc09-41ea-8f12-eeac74786df2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:58.894880 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c1d2b70f-dc09-41ea-8f12-eeac74786df2 admin admin] The token cache is being invalidate because project 7016950331214535a290476500499e48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:08:58.895324 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c1d2b70f-dc09-41ea-8f12-eeac74786df2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:58.896091 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c1d2b70f-dc09-41ea-8f12-eeac74786df2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:58.899367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 943/1857] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:08:58 2022] DELETE /identity/v3/projects/7016950331214535a290476500499e48 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:58.902523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dd775-a54c-4afd-960b-3685db6de7e4 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:58.913143 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dd775-a54c-4afd-960b-3685db6de7e4 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:58.917494 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:08:58.978334 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97a6d3-d94b-4b70-b022-d5e174417dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:58.979278 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97a6d3-d94b-4b70-b022-d5e174417dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:58.979568 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97a6d3-d94b-4b70-b022-d5e174417dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:58.979862 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97a6d3-d94b-4b70-b022-d5e174417dac admin admin] PATH_INFO: `/v3/users/7d66ff536bdd4e53b84733439ce50d30` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:58.981106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97a6d3-d94b-4b70-b022-d5e174417dac admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d66ff536bdd4e53b84733439ce50d30)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:58.983347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97a6d3-d94b-4b70-b022-d5e174417dac admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:08:59.182574 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3e97a6d3-d94b-4b70-b022-d5e174417dac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d66ff536bdd4e53b84733439ce50d30'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:59.192912 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3e97a6d3-d94b-4b70-b022-d5e174417dac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d66ff536bdd4e53b84733439ce50d30'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:59.193633 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3e97a6d3-d94b-4b70-b022-d5e174417dac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d66ff536bdd4e53b84733439ce50d30'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:59.194241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3e97a6d3-d94b-4b70-b022-d5e174417dac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d66ff536bdd4e53b84733439ce50d30'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:59.207523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3e97a6d3-d94b-4b70-b022-d5e174417dac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d66ff536bdd4e53b84733439ce50d30'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:08:59.293578 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 915/1858] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:08:58 2022] DELETE /identity/v3/users/7d66ff536bdd4e53b84733439ce50d30 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:08:59.863478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2b70f-dc09-41ea-8f12-eeac74786df2 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:08:59.865147 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c1d2b70f-dc09-41ea-8f12-eeac74786df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:08:59.879127 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2b70f-dc09-41ea-8f12-eeac74786df2 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:08:59.989243 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5685790-31b5-479c-a1d1-f2b381543e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:08:59.990072 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5685790-31b5-479c-a1d1-f2b381543e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:08:59.990548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5685790-31b5-479c-a1d1-f2b381543e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:08:59.991505 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5685790-31b5-479c-a1d1-f2b381543e37 admin admin] PATH_INFO: `/v3/projects/4aac0577bf2f4a829d43f3e3d3c8d29b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:08:59.993376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5685790-31b5-479c-a1d1-f2b381543e37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aac0577bf2f4a829d43f3e3d3c8d29b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:08:59.996090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5685790-31b5-479c-a1d1-f2b381543e37 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:00.106606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b5685790-31b5-479c-a1d1-f2b381543e37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aac0577bf2f4a829d43f3e3d3c8d29b', 'request_id': 'req-b5685790-31b5-479c-a1d1-f2b381543e37'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:00.195748 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b5685790-31b5-479c-a1d1-f2b381543e37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aac0577bf2f4a829d43f3e3d3c8d29b', 'request_id': 'req-b5685790-31b5-479c-a1d1-f2b381543e37'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:00.204628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b5685790-31b5-479c-a1d1-f2b381543e37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aac0577bf2f4a829d43f3e3d3c8d29b', 'request_id': 'req-b5685790-31b5-479c-a1d1-f2b381543e37'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:00.217952 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b5685790-31b5-479c-a1d1-f2b381543e37 admin admin] The token cache is being invalidate because project 4aac0577bf2f4a829d43f3e3d3c8d29b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:00.218231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b5685790-31b5-479c-a1d1-f2b381543e37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:00.218929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b5685790-31b5-479c-a1d1-f2b381543e37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:00.220658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 944/1859] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:08:59 2022] DELETE /identity/v3/projects/4aac0577bf2f4a829d43f3e3d3c8d29b => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:00.224769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97a6d3-d94b-4b70-b022-d5e174417dac admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:00.229289 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3e97a6d3-d94b-4b70-b022-d5e174417dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:00.239843 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97a6d3-d94b-4b70-b022-d5e174417dac admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:00.305797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd6e16-42ba-4d3d-9cae-da296bdc417d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:00.306688 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd6e16-42ba-4d3d-9cae-da296bdc417d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:00.306978 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd6e16-42ba-4d3d-9cae-da296bdc417d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:00.307306 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd6e16-42ba-4d3d-9cae-da296bdc417d admin admin] PATH_INFO: `/v3/users/81210649f5ac4a8da32c9fad677f663d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:00.308516 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd6e16-42ba-4d3d-9cae-da296bdc417d admin admin] RBAC: Authorizing `identity:delete_user(user_id=81210649f5ac4a8da32c9fad677f663d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:00.311031 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd6e16-42ba-4d3d-9cae-da296bdc417d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:00.473464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:00.491313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-61bd6e16-42ba-4d3d-9cae-da296bdc417d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81210649f5ac4a8da32c9fad677f663d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:00.504423 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-61bd6e16-42ba-4d3d-9cae-da296bdc417d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81210649f5ac4a8da32c9fad677f663d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:00.505514 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-61bd6e16-42ba-4d3d-9cae-da296bdc417d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81210649f5ac4a8da32c9fad677f663d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:00.506453 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-61bd6e16-42ba-4d3d-9cae-da296bdc417d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81210649f5ac4a8da32c9fad677f663d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:00.533976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-61bd6e16-42ba-4d3d-9cae-da296bdc417d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81210649f5ac4a8da32c9fad677f663d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:00.637207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 916/1860] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:00 2022] DELETE /identity/v3/users/81210649f5ac4a8da32c9fad677f663d => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:01.166070 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5685790-31b5-479c-a1d1-f2b381543e37 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:01.167635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b5685790-31b5-479c-a1d1-f2b381543e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:01.182948 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5685790-31b5-479c-a1d1-f2b381543e37 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:01.255044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:01.275443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca6a0e-41d1-45bb-b096-8e665281f5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:01.276246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca6a0e-41d1-45bb-b096-8e665281f5b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:01.276531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca6a0e-41d1-45bb-b096-8e665281f5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:01.276810 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca6a0e-41d1-45bb-b096-8e665281f5b8 admin admin] PATH_INFO: `/v3/projects/0ca976e996c549d5a8a216c4c906e18b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:01.279106 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca6a0e-41d1-45bb-b096-8e665281f5b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ca976e996c549d5a8a216c4c906e18b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:01.281685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca6a0e-41d1-45bb-b096-8e665281f5b8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:01.394985 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-86ca6a0e-41d1-45bb-b096-8e665281f5b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ca976e996c549d5a8a216c4c906e18b', 'request_id': 'req-86ca6a0e-41d1-45bb-b096-8e665281f5b8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:01.459300 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-86ca6a0e-41d1-45bb-b096-8e665281f5b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ca976e996c549d5a8a216c4c906e18b', 'request_id': 'req-86ca6a0e-41d1-45bb-b096-8e665281f5b8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:01.466269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-86ca6a0e-41d1-45bb-b096-8e665281f5b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ca976e996c549d5a8a216c4c906e18b', 'request_id': 'req-86ca6a0e-41d1-45bb-b096-8e665281f5b8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:01.477607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-86ca6a0e-41d1-45bb-b096-8e665281f5b8 admin admin] The token cache is being invalidate because project 0ca976e996c549d5a8a216c4c906e18b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:01.477902 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-86ca6a0e-41d1-45bb-b096-8e665281f5b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:01.478604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-86ca6a0e-41d1-45bb-b096-8e665281f5b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:01.480215 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 945/1861] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:01 2022] DELETE /identity/v3/projects/0ca976e996c549d5a8a216c4c906e18b => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:01.486239 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ddc16-6a08-4db1-abc1-d5c3f2292c68 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:01.486764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ddc16-6a08-4db1-abc1-d5c3f2292c68 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:01.487251 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ddc16-6a08-4db1-abc1-d5c3f2292c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:01.505870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-1c4ddc16-6a08-4db1-abc1-d5c3f2292c68 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:01.507085 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1c4ddc16-6a08-4db1-abc1-d5c3f2292c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:01.569611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 917/1862] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:01.576973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b856f-a0a0-4d0b-96e9-94fb28d977f8 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:01.577264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b856f-a0a0-4d0b-96e9-94fb28d977f8 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:01.577543 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b856f-a0a0-4d0b-96e9-94fb28d977f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:01.597034 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-473b856f-a0a0-4d0b-96e9-94fb28d977f8 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:01.598492 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-473b856f-a0a0-4d0b-96e9-94fb28d977f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:01.608714 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 946/1863] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:09:01.613216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ddc16-6a08-4db1-abc1-d5c3f2292c68 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:01.635319 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ddc16-6a08-4db1-abc1-d5c3f2292c68 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:01.643513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31537ad-207b-457f-b606-83101bbaee5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:01.644496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31537ad-207b-457f-b606-83101bbaee5b admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:01.644950 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31537ad-207b-457f-b606-83101bbaee5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:01.645363 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31537ad-207b-457f-b606-83101bbaee5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:01.646500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31537ad-207b-457f-b606-83101bbaee5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:09:01.646957 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31537ad-207b-457f-b606-83101bbaee5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:01.649617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31537ad-207b-457f-b606-83101bbaee5b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:01.669266 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 918/1864] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:09:01.673547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b856f-a0a0-4d0b-96e9-94fb28d977f8 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:01.683040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b856f-a0a0-4d0b-96e9-94fb28d977f8 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:01.688387 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8f873-835d-4df1-80b2-ddde2eb0088d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:01.689044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8f873-835d-4df1-80b2-ddde2eb0088d admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:01.689334 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8f873-835d-4df1-80b2-ddde2eb0088d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:01.689614 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8f873-835d-4df1-80b2-ddde2eb0088d admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:01.690403 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8f873-835d-4df1-80b2-ddde2eb0088d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:01.692293 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8f873-835d-4df1-80b2-ddde2eb0088d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:01.775887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 947/1865] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:01 2022] POST /identity/v3/projects => generated 401 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:01.780498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31537ad-207b-457f-b606-83101bbaee5b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:01.790862 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31537ad-207b-457f-b606-83101bbaee5b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:01.849961 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db80df-0137-4694-8c75-b25bb3405773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:01.850651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db80df-0137-4694-8c75-b25bb3405773 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:01.850947 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db80df-0137-4694-8c75-b25bb3405773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:01.851293 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db80df-0137-4694-8c75-b25bb3405773 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:01.852337 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db80df-0137-4694-8c75-b25bb3405773 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:01.854351 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db80df-0137-4694-8c75-b25bb3405773 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:01.956817 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 919/1866] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:01 2022] POST /identity/v3/users => generated 301 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:01.961166 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8f873-835d-4df1-80b2-ddde2eb0088d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:01.972429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8f873-835d-4df1-80b2-ddde2eb0088d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:01.977182 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae53374-4cbb-4e6c-a1df-8c7e642f54a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:01.977857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae53374-4cbb-4e6c-a1df-8c7e642f54a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:01.983051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae53374-4cbb-4e6c-a1df-8c7e642f54a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:01.983051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae53374-4cbb-4e6c-a1df-8c7e642f54a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:01.983051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae53374-4cbb-4e6c-a1df-8c7e642f54a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:01.983051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae53374-4cbb-4e6c-a1df-8c7e642f54a5 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:01.992032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 948/1867] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:01.996315 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db80df-0137-4694-8c75-b25bb3405773 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:02.007297 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db80df-0137-4694-8c75-b25bb3405773 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:02.011842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401b2db-4282-4932-806e-09afad44f4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:02.012184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401b2db-4282-4932-806e-09afad44f4cf admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:02.012476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401b2db-4282-4932-806e-09afad44f4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:02.012760 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401b2db-4282-4932-806e-09afad44f4cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:02.013486 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401b2db-4282-4932-806e-09afad44f4cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:02.015091 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401b2db-4282-4932-806e-09afad44f4cf admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:02.025696 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 920/1868] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:02.030859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae53374-4cbb-4e6c-a1df-8c7e642f54a5 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:02.041121 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae53374-4cbb-4e6c-a1df-8c7e642f54a5 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:02.046329 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf09f4-5c80-4386-ac57-02a5feae767d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:02.047243 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf09f4-5c80-4386-ac57-02a5feae767d admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:02.047532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf09f4-5c80-4386-ac57-02a5feae767d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:02.047798 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf09f4-5c80-4386-ac57-02a5feae767d admin admin] PATH_INFO: `/v3/projects/93edbffa75e7420e8ddd6f317ba6a380/users/329fd6d7251b4873b9a1c7d9549d2cad/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:02.062327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf09f4-5c80-4386-ac57-02a5feae767d admin admin] RBAC: Authorizing `identity:create_grant(project_id=93edbffa75e7420e8ddd6f317ba6a380, user_id=329fd6d7251b4873b9a1c7d9549d2cad, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:02.065068 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf09f4-5c80-4386-ac57-02a5feae767d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:02.151968 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 949/1869] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:02 2022] PUT /identity/v3/projects/93edbffa75e7420e8ddd6f317ba6a380/users/329fd6d7251b4873b9a1c7d9549d2cad/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:02.169364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332cf09-9892-43fe-b50c-8459f7cb01ba None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:02.170471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332cf09-9892-43fe-b50c-8459f7cb01ba None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:02.171157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332cf09-9892-43fe-b50c-8459f7cb01ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:02.209933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:02.214271 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c332cf09-9892-43fe-b50c-8459f7cb01ba None None] MFA Rules not processed for user `329fd6d7251b4873b9a1c7d9549d2cad`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:02.217184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c332cf09-9892-43fe-b50c-8459f7cb01ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:02.311474 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 921/1870] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:02 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:09:02.313611 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf09f4-5c80-4386-ac57-02a5feae767d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:02.384601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf09f4-5c80-4386-ac57-02a5feae767d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:02.392618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a693b-8947-4090-ae65-bf2a3650a7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:02.392618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a693b-8947-4090-ae65-bf2a3650a7ce admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:02.394790 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a693b-8947-4090-ae65-bf2a3650a7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:02.394790 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a693b-8947-4090-ae65-bf2a3650a7ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:02.395305 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a693b-8947-4090-ae65-bf2a3650a7ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:02.397627 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a693b-8947-4090-ae65-bf2a3650a7ce admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:02.419628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:02.553569 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 950/1871] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:02 2022] POST /identity/v3/projects => generated 401 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:02.564917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332cf09-9892-43fe-b50c-8459f7cb01ba None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:02.577824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332cf09-9892-43fe-b50c-8459f7cb01ba None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:02.642927 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abba635-67fb-4e1e-a216-35877a0eefb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:02.643545 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abba635-67fb-4e1e-a216-35877a0eefb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:02.643835 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abba635-67fb-4e1e-a216-35877a0eefb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:02.644115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abba635-67fb-4e1e-a216-35877a0eefb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:02.644824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abba635-67fb-4e1e-a216-35877a0eefb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:02.646810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abba635-67fb-4e1e-a216-35877a0eefb6 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:02.757180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 922/1872] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:02 2022] POST /identity/v3/users => generated 307 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:02.765342 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a693b-8947-4090-ae65-bf2a3650a7ce admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:02.783930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a693b-8947-4090-ae65-bf2a3650a7ce admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:02.790355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4d878-667c-4c7a-a442-873d9a0ef720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:02.791426 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4d878-667c-4c7a-a442-873d9a0ef720 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:02.791712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4d878-667c-4c7a-a442-873d9a0ef720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:02.791989 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4d878-667c-4c7a-a442-873d9a0ef720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:02.792958 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4d878-667c-4c7a-a442-873d9a0ef720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:02.794629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4d878-667c-4c7a-a442-873d9a0ef720 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:02.806453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 951/1873] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:09:02.811747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abba635-67fb-4e1e-a216-35877a0eefb6 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:02.822031 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abba635-67fb-4e1e-a216-35877a0eefb6 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:02.827611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8daa08-7d6d-47cd-8721-8dad738fc1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:02.828514 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8daa08-7d6d-47cd-8721-8dad738fc1a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:02.828793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8daa08-7d6d-47cd-8721-8dad738fc1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:02.829073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8daa08-7d6d-47cd-8721-8dad738fc1a3 admin admin] PATH_INFO: `/v3/projects/236ebed18363417bbbd62af0c881a25b/users/2a46023e987c402c86f55e4fe84eae65/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:02.845860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8daa08-7d6d-47cd-8721-8dad738fc1a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=236ebed18363417bbbd62af0c881a25b, user_id=2a46023e987c402c86f55e4fe84eae65, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:02.848733 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8daa08-7d6d-47cd-8721-8dad738fc1a3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:02.946862 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 923/1874] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:02 2022] PUT /identity/v3/projects/236ebed18363417bbbd62af0c881a25b/users/2a46023e987c402c86f55e4fe84eae65/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:02.950963 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4d878-667c-4c7a-a442-873d9a0ef720 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:02.962841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4d878-667c-4c7a-a442-873d9a0ef720 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:03.035741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74d925-90d7-4474-b1bc-fc717089740c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:03.036425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74d925-90d7-4474-b1bc-fc717089740c admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:03.036710 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74d925-90d7-4474-b1bc-fc717089740c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:03.036991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74d925-90d7-4474-b1bc-fc717089740c admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:03.037713 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74d925-90d7-4474-b1bc-fc717089740c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:03.039357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74d925-90d7-4474-b1bc-fc717089740c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:03.050099 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 952/1875] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:03.054848 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8daa08-7d6d-47cd-8721-8dad738fc1a3 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:03.065438 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8daa08-7d6d-47cd-8721-8dad738fc1a3 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:03.070767 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce8c12-9b5d-4908-9162-7a49ea7d9417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:03.071698 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce8c12-9b5d-4908-9162-7a49ea7d9417 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:03.071980 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce8c12-9b5d-4908-9162-7a49ea7d9417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:03.072260 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce8c12-9b5d-4908-9162-7a49ea7d9417 admin admin] PATH_INFO: `/v3/projects/236ebed18363417bbbd62af0c881a25b/users/2a46023e987c402c86f55e4fe84eae65/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:03.074234 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce8c12-9b5d-4908-9162-7a49ea7d9417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=236ebed18363417bbbd62af0c881a25b, user_id=2a46023e987c402c86f55e4fe84eae65, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:03.077542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce8c12-9b5d-4908-9162-7a49ea7d9417 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:03.088485 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 924/1876] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:03 2022] PUT /identity/v3/projects/236ebed18363417bbbd62af0c881a25b/users/2a46023e987c402c86f55e4fe84eae65/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:03.099934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f4baa-71dc-45a8-bde7-a6bd9705193d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:03.100231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f4baa-71dc-45a8-bde7-a6bd9705193d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:03.100523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f4baa-71dc-45a8-bde7-a6bd9705193d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:03.136043 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-8c3f4baa-71dc-45a8-bde7-a6bd9705193d None None] MFA Rules not processed for user `2a46023e987c402c86f55e4fe84eae65`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:03.137337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8c3f4baa-71dc-45a8-bde7-a6bd9705193d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:03.241654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 953/1877] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:03 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:03.254057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce8c12-9b5d-4908-9162-7a49ea7d9417 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:03.259239 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-81ce8c12-9b5d-4908-9162-7a49ea7d9417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:03.332935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce8c12-9b5d-4908-9162-7a49ea7d9417 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:03.338930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e685e6e-7986-48e2-966f-31585306690b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:03.339964 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e685e6e-7986-48e2-966f-31585306690b service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:03.340407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e685e6e-7986-48e2-966f-31585306690b service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:03.341305 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e685e6e-7986-48e2-966f-31585306690b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:03.357527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e685e6e-7986-48e2-966f-31585306690b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:03.359405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e685e6e-7986-48e2-966f-31585306690b service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:03.365185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:03.376635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 925/1878] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:03 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:03.664321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f4baa-71dc-45a8-bde7-a6bd9705193d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:03.682335 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f4baa-71dc-45a8-bde7-a6bd9705193d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:03.690721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0419e0-79d9-4307-9f43-3505935af213 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:03.692132 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0419e0-79d9-4307-9f43-3505935af213 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:03.692772 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0419e0-79d9-4307-9f43-3505935af213 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:03.693435 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0419e0-79d9-4307-9f43-3505935af213 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:03.782161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0419e0-79d9-4307-9f43-3505935af213 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:03.784984 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0419e0-79d9-4307-9f43-3505935af213 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:03.796664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 954/1879] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:03 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:09.485567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e685e6e-7986-48e2-966f-31585306690b service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:09.487066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4e685e6e-7986-48e2-966f-31585306690b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:09.568501 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e685e6e-7986-48e2-966f-31585306690b service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:09.580617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c9556-34e4-4dd5-8a42-5a24c14c00b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:09.581687 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c9556-34e4-4dd5-8a42-5a24c14c00b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:09.582058 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c9556-34e4-4dd5-8a42-5a24c14c00b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:09.582511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c9556-34e4-4dd5-8a42-5a24c14c00b1 admin admin] PATH_INFO: `/v3/users/aed2e1102ad341f2a34144e4f9ce5800` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:09.591516 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c9556-34e4-4dd5-8a42-5a24c14c00b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aed2e1102ad341f2a34144e4f9ce5800)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:09.591516 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c9556-34e4-4dd5-8a42-5a24c14c00b1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:09.818373 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9a5c9556-34e4-4dd5-8a42-5a24c14c00b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aed2e1102ad341f2a34144e4f9ce5800'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:09.837302 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9a5c9556-34e4-4dd5-8a42-5a24c14c00b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aed2e1102ad341f2a34144e4f9ce5800'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:09.837302 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9a5c9556-34e4-4dd5-8a42-5a24c14c00b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aed2e1102ad341f2a34144e4f9ce5800'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:09.837302 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9a5c9556-34e4-4dd5-8a42-5a24c14c00b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aed2e1102ad341f2a34144e4f9ce5800'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:09.858770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9a5c9556-34e4-4dd5-8a42-5a24c14c00b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aed2e1102ad341f2a34144e4f9ce5800'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:09.957098 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 926/1880] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:09 2022] DELETE /identity/v3/users/aed2e1102ad341f2a34144e4f9ce5800 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:09.957619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4064d5d-92d2-4dc4-a55a-6f6db5414e42 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:09.958097 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4064d5d-92d2-4dc4-a55a-6f6db5414e42 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:09.958273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4064d5d-92d2-4dc4-a55a-6f6db5414e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:09.966338 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:09.987911 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c4064d5d-92d2-4dc4-a55a-6f6db5414e42 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:09.989984 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c4064d5d-92d2-4dc4-a55a-6f6db5414e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:10.006505 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c9556-34e4-4dd5-8a42-5a24c14c00b1 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:10.008637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9a5c9556-34e4-4dd5-8a42-5a24c14c00b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:10.105586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c9556-34e4-4dd5-8a42-5a24c14c00b1 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:10.111077 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0014fa2-64d3-49b9-ac6d-fa771eb73866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:10.112012 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0014fa2-64d3-49b9-ac6d-fa771eb73866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:10.112303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0014fa2-64d3-49b9-ac6d-fa771eb73866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:10.113532 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0014fa2-64d3-49b9-ac6d-fa771eb73866 admin admin] PATH_INFO: `/v3/users/cd25ff075bc244b799bebac3a2889b65` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:10.115521 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0014fa2-64d3-49b9-ac6d-fa771eb73866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd25ff075bc244b799bebac3a2889b65)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:10.115904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0014fa2-64d3-49b9-ac6d-fa771eb73866 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:10.119912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 955/1881] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:09:10.121296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4064d5d-92d2-4dc4-a55a-6f6db5414e42 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:10.123620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c4064d5d-92d2-4dc4-a55a-6f6db5414e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:10.139964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4064d5d-92d2-4dc4-a55a-6f6db5414e42 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:10.145790 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff8a19-0f21-43a9-ad39-5ce7d88826ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:10.150055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff8a19-0f21-43a9-ad39-5ce7d88826ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:10.150055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff8a19-0f21-43a9-ad39-5ce7d88826ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:10.150055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff8a19-0f21-43a9-ad39-5ce7d88826ff admin admin] PATH_INFO: `/v3/users/329fd6d7251b4873b9a1c7d9549d2cad` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:10.150404 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff8a19-0f21-43a9-ad39-5ce7d88826ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=329fd6d7251b4873b9a1c7d9549d2cad)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:10.152457 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff8a19-0f21-43a9-ad39-5ce7d88826ff admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:10.178779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:10.289256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e0014fa2-64d3-49b9-ac6d-fa771eb73866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd25ff075bc244b799bebac3a2889b65'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:10.296378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e0014fa2-64d3-49b9-ac6d-fa771eb73866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd25ff075bc244b799bebac3a2889b65'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:10.297112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e0014fa2-64d3-49b9-ac6d-fa771eb73866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd25ff075bc244b799bebac3a2889b65'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:10.297766 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e0014fa2-64d3-49b9-ac6d-fa771eb73866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd25ff075bc244b799bebac3a2889b65'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:10.310072 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e0014fa2-64d3-49b9-ac6d-fa771eb73866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd25ff075bc244b799bebac3a2889b65'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:10.343890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-85ff8a19-0f21-43a9-ad39-5ce7d88826ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '329fd6d7251b4873b9a1c7d9549d2cad'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:10.344557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-85ff8a19-0f21-43a9-ad39-5ce7d88826ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '329fd6d7251b4873b9a1c7d9549d2cad'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:10.359067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-85ff8a19-0f21-43a9-ad39-5ce7d88826ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '329fd6d7251b4873b9a1c7d9549d2cad'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:10.359775 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-85ff8a19-0f21-43a9-ad39-5ce7d88826ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '329fd6d7251b4873b9a1c7d9549d2cad'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:10.368256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-85ff8a19-0f21-43a9-ad39-5ce7d88826ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '329fd6d7251b4873b9a1c7d9549d2cad'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:10.384384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 927/1882] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:10 2022] DELETE /identity/v3/users/cd25ff075bc244b799bebac3a2889b65 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:10.385460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0014fa2-64d3-49b9-ac6d-fa771eb73866 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:10.387790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e0014fa2-64d3-49b9-ac6d-fa771eb73866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:10.449850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0014fa2-64d3-49b9-ac6d-fa771eb73866 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:10.453676 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f9ca9-25c1-4747-a984-284974e97723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:10.454369 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f9ca9-25c1-4747-a984-284974e97723 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:10.454661 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f9ca9-25c1-4747-a984-284974e97723 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:10.454946 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f9ca9-25c1-4747-a984-284974e97723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:10.457250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a25f9ca9-25c1-4747-a984-284974e97723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:10.465488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 956/1883] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:10 2022] DELETE /identity/v3/users/329fd6d7251b4873b9a1c7d9549d2cad => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:10.467884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feba3bd-983f-4aa2-b9cc-366adb32157d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:10.469800 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feba3bd-983f-4aa2-b9cc-366adb32157d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:10.470293 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feba3bd-983f-4aa2-b9cc-366adb32157d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:10.490176 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3feba3bd-983f-4aa2-b9cc-366adb32157d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:10.491591 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3feba3bd-983f-4aa2-b9cc-366adb32157d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:10.561586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 957/1884] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:10.563119 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323c875-925b-4d86-bcdf-8cb1a94a93a6 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:10.563448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323c875-925b-4d86-bcdf-8cb1a94a93a6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:10.565607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323c875-925b-4d86-bcdf-8cb1a94a93a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:10.565820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f9ca9-25c1-4747-a984-284974e97723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:10.565820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f9ca9-25c1-4747-a984-284974e97723 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:10.578677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 928/1885] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:10.579566 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f9ca9-25c1-4747-a984-284974e97723 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:10.580935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a25f9ca9-25c1-4747-a984-284974e97723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:10.582630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-4323c875-925b-4d86-bcdf-8cb1a94a93a6 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:10.584147 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4323c875-925b-4d86-bcdf-8cb1a94a93a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:10.593841 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f9ca9-25c1-4747-a984-284974e97723 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:10.596040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 958/1886] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:10.624278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323c875-925b-4d86-bcdf-8cb1a94a93a6 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:10.626319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4323c875-925b-4d86-bcdf-8cb1a94a93a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:10.646704 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:10.671619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:10.680101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e8638-6e4d-42d2-b85d-852eb3c02afd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:10.681490 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e8638-6e4d-42d2-b85d-852eb3c02afd service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:10.681856 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e8638-6e4d-42d2-b85d-852eb3c02afd service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:10.682209 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e8638-6e4d-42d2-b85d-852eb3c02afd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:10.708023 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e8638-6e4d-42d2-b85d-852eb3c02afd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:10.710919 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e8638-6e4d-42d2-b85d-852eb3c02afd service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:10.722887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323c875-925b-4d86-bcdf-8cb1a94a93a6 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:10.734536 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 929/1887] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:10.741256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c973437-b2d8-4e34-b90b-3455f83b8d9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:10.743135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c973437-b2d8-4e34-b90b-3455f83b8d9e service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:10.745577 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c973437-b2d8-4e34-b90b-3455f83b8d9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:10.745856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c973437-b2d8-4e34-b90b-3455f83b8d9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:10.769644 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c973437-b2d8-4e34-b90b-3455f83b8d9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:10.775441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c973437-b2d8-4e34-b90b-3455f83b8d9e service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:10.791731 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 959/1888] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:11.399624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e8638-6e4d-42d2-b85d-852eb3c02afd service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:11.401347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-bc2e8638-6e4d-42d2-b85d-852eb3c02afd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:11.407752 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c973437-b2d8-4e34-b90b-3455f83b8d9e service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:11.409427 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9c973437-b2d8-4e34-b90b-3455f83b8d9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:11.429070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e8638-6e4d-42d2-b85d-852eb3c02afd service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:11.431340 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c973437-b2d8-4e34-b90b-3455f83b8d9e service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:11.439719 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98e5ca-9608-4f7e-af7b-855b2910e2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:11.442693 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4827ea-1064-46ca-b371-e57e1d4b1f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:11.443761 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4827ea-1064-46ca-b371-e57e1d4b1f73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:11.444097 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4827ea-1064-46ca-b371-e57e1d4b1f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:11.444546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4827ea-1064-46ca-b371-e57e1d4b1f73 admin admin] PATH_INFO: `/v3/projects/40d521bd6dbd470f9739c7d5c40863b8` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:11.448017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4827ea-1064-46ca-b371-e57e1d4b1f73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40d521bd6dbd470f9739c7d5c40863b8)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:11.456770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4827ea-1064-46ca-b371-e57e1d4b1f73 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:11.466694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98e5ca-9608-4f7e-af7b-855b2910e2dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:11.467291 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98e5ca-9608-4f7e-af7b-855b2910e2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:11.469640 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98e5ca-9608-4f7e-af7b-855b2910e2dd admin admin] PATH_INFO: `/v3/projects/faf21454928a4888b102f0683c19cff6` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:11.475713 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98e5ca-9608-4f7e-af7b-855b2910e2dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=faf21454928a4888b102f0683c19cff6)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:11.485256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98e5ca-9608-4f7e-af7b-855b2910e2dd admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:11.596008 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8b4827ea-1064-46ca-b371-e57e1d4b1f73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40d521bd6dbd470f9739c7d5c40863b8', 'request_id': 'req-8b4827ea-1064-46ca-b371-e57e1d4b1f73'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:11.629739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-aa98e5ca-9608-4f7e-af7b-855b2910e2dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faf21454928a4888b102f0683c19cff6', 'request_id': 'req-aa98e5ca-9608-4f7e-af7b-855b2910e2dd'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:11.681624 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8b4827ea-1064-46ca-b371-e57e1d4b1f73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40d521bd6dbd470f9739c7d5c40863b8', 'request_id': 'req-8b4827ea-1064-46ca-b371-e57e1d4b1f73'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:11.690108 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8b4827ea-1064-46ca-b371-e57e1d4b1f73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40d521bd6dbd470f9739c7d5c40863b8', 'request_id': 'req-8b4827ea-1064-46ca-b371-e57e1d4b1f73'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:11.703262 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8b4827ea-1064-46ca-b371-e57e1d4b1f73 admin admin] The token cache is being invalidate because project 40d521bd6dbd470f9739c7d5c40863b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:11.703601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8b4827ea-1064-46ca-b371-e57e1d4b1f73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:11.704321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8b4827ea-1064-46ca-b371-e57e1d4b1f73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:11.706044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 960/1889] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:11 2022] DELETE /identity/v3/projects/40d521bd6dbd470f9739c7d5c40863b8 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:11.706871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4827ea-1064-46ca-b371-e57e1d4b1f73 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:11.708754 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8b4827ea-1064-46ca-b371-e57e1d4b1f73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:11.733542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-aa98e5ca-9608-4f7e-af7b-855b2910e2dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faf21454928a4888b102f0683c19cff6', 'request_id': 'req-aa98e5ca-9608-4f7e-af7b-855b2910e2dd'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:11.740957 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-aa98e5ca-9608-4f7e-af7b-855b2910e2dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faf21454928a4888b102f0683c19cff6', 'request_id': 'req-aa98e5ca-9608-4f7e-af7b-855b2910e2dd'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:11.755597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-aa98e5ca-9608-4f7e-af7b-855b2910e2dd admin admin] The token cache is being invalidate because project faf21454928a4888b102f0683c19cff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:11.756068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-aa98e5ca-9608-4f7e-af7b-855b2910e2dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:11.757060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-aa98e5ca-9608-4f7e-af7b-855b2910e2dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:11.759734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 930/1890] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:11 2022] DELETE /identity/v3/projects/faf21454928a4888b102f0683c19cff6 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:11.760594 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98e5ca-9608-4f7e-af7b-855b2910e2dd admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:11.762048 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-aa98e5ca-9608-4f7e-af7b-855b2910e2dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:11.793133 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4827ea-1064-46ca-b371-e57e1d4b1f73 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:11.860562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98e5ca-9608-4f7e-af7b-855b2910e2dd admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:11.860875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:11.867928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94db36-f624-4ed4-8159-fec725c6a0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:11.868871 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94db36-f624-4ed4-8159-fec725c6a0a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:11.869160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94db36-f624-4ed4-8159-fec725c6a0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:11.872976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94db36-f624-4ed4-8159-fec725c6a0a6 admin admin] PATH_INFO: `/v3/users/553b440942d345478336133e77492bb4` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:11.872976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd94db36-f624-4ed4-8159-fec725c6a0a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=553b440942d345478336133e77492bb4)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:11.873437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd94db36-f624-4ed4-8159-fec725c6a0a6 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:11.878793 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedb65a-5a92-463c-a199-51c4d3de0907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:11.879671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedb65a-5a92-463c-a199-51c4d3de0907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:11.879977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedb65a-5a92-463c-a199-51c4d3de0907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:11.880290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedb65a-5a92-463c-a199-51c4d3de0907 admin admin] PATH_INFO: `/v3/projects/93edbffa75e7420e8ddd6f317ba6a380` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:11.881473 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedb65a-5a92-463c-a199-51c4d3de0907 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93edbffa75e7420e8ddd6f317ba6a380)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:11.883455 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedb65a-5a92-463c-a199-51c4d3de0907 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:12.064627 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4bedb65a-5a92-463c-a199-51c4d3de0907 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93edbffa75e7420e8ddd6f317ba6a380', 'request_id': 'req-4bedb65a-5a92-463c-a199-51c4d3de0907'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.066013 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-bd94db36-f624-4ed4-8159-fec725c6a0a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '553b440942d345478336133e77492bb4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.077317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-bd94db36-f624-4ed4-8159-fec725c6a0a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '553b440942d345478336133e77492bb4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.078083 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-bd94db36-f624-4ed4-8159-fec725c6a0a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '553b440942d345478336133e77492bb4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.078725 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-bd94db36-f624-4ed4-8159-fec725c6a0a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '553b440942d345478336133e77492bb4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.095183 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-bd94db36-f624-4ed4-8159-fec725c6a0a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '553b440942d345478336133e77492bb4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.097021 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:12.161981 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4bedb65a-5a92-463c-a199-51c4d3de0907 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93edbffa75e7420e8ddd6f317ba6a380', 'request_id': 'req-4bedb65a-5a92-463c-a199-51c4d3de0907'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.169123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4bedb65a-5a92-463c-a199-51c4d3de0907 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93edbffa75e7420e8ddd6f317ba6a380', 'request_id': 'req-4bedb65a-5a92-463c-a199-51c4d3de0907'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.180708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4bedb65a-5a92-463c-a199-51c4d3de0907 admin admin] The token cache is being invalidate because project 93edbffa75e7420e8ddd6f317ba6a380 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:12.180997 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4bedb65a-5a92-463c-a199-51c4d3de0907 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.181679 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4bedb65a-5a92-463c-a199-51c4d3de0907 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.183296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 961/1891] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:11 2022] DELETE /identity/v3/projects/93edbffa75e7420e8ddd6f317ba6a380 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:12.184731 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedb65a-5a92-463c-a199-51c4d3de0907 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:12.186030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4bedb65a-5a92-463c-a199-51c4d3de0907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:12.252975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedb65a-5a92-463c-a199-51c4d3de0907 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:12.255928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 931/1892] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:11 2022] DELETE /identity/v3/users/553b440942d345478336133e77492bb4 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:12.263349 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb920f22-af3d-4065-a7a4-5c57b3347c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:12.263349 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb920f22-af3d-4065-a7a4-5c57b3347c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:12.263349 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb920f22-af3d-4065-a7a4-5c57b3347c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:12.263349 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb920f22-af3d-4065-a7a4-5c57b3347c36 admin admin] PATH_INFO: `/v3/users/f6464d9246364819a85f0b7e3679a83a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:12.264219 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94db36-f624-4ed4-8159-fec725c6a0a6 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:12.264301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb920f22-af3d-4065-a7a4-5c57b3347c36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6464d9246364819a85f0b7e3679a83a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:12.264382 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-bd94db36-f624-4ed4-8159-fec725c6a0a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:12.264454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb920f22-af3d-4065-a7a4-5c57b3347c36 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:12.344041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94db36-f624-4ed4-8159-fec725c6a0a6 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:12.351522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab487f01-f2e7-4528-a3b4-0e753094ec3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:12.352463 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab487f01-f2e7-4528-a3b4-0e753094ec3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:12.352946 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab487f01-f2e7-4528-a3b4-0e753094ec3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:12.353453 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab487f01-f2e7-4528-a3b4-0e753094ec3d admin admin] PATH_INFO: `/v3/users/2a46023e987c402c86f55e4fe84eae65` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:12.356164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab487f01-f2e7-4528-a3b4-0e753094ec3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a46023e987c402c86f55e4fe84eae65)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:12.358204 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab487f01-f2e7-4528-a3b4-0e753094ec3d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:12.413442 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fb920f22-af3d-4065-a7a4-5c57b3347c36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6464d9246364819a85f0b7e3679a83a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.414353 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fb920f22-af3d-4065-a7a4-5c57b3347c36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6464d9246364819a85f0b7e3679a83a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.432204 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fb920f22-af3d-4065-a7a4-5c57b3347c36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6464d9246364819a85f0b7e3679a83a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.433417 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fb920f22-af3d-4065-a7a4-5c57b3347c36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6464d9246364819a85f0b7e3679a83a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.443488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fb920f22-af3d-4065-a7a4-5c57b3347c36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6464d9246364819a85f0b7e3679a83a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.530962 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 962/1893] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:12 2022] DELETE /identity/v3/users/f6464d9246364819a85f0b7e3679a83a => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:12.568802 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ab487f01-f2e7-4528-a3b4-0e753094ec3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a46023e987c402c86f55e4fe84eae65'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.579719 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ab487f01-f2e7-4528-a3b4-0e753094ec3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a46023e987c402c86f55e4fe84eae65'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.595375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ab487f01-f2e7-4528-a3b4-0e753094ec3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a46023e987c402c86f55e4fe84eae65'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.600936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ab487f01-f2e7-4528-a3b4-0e753094ec3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a46023e987c402c86f55e4fe84eae65'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.642971 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ab487f01-f2e7-4528-a3b4-0e753094ec3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a46023e987c402c86f55e4fe84eae65'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:12.760760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:12.774182 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 932/1894] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:12 2022] DELETE /identity/v3/users/2a46023e987c402c86f55e4fe84eae65 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:12.876442 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb920f22-af3d-4065-a7a4-5c57b3347c36 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:12.881531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-fb920f22-af3d-4065-a7a4-5c57b3347c36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:13.000820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb920f22-af3d-4065-a7a4-5c57b3347c36 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:13.094624 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20baed06-d307-4c68-b9b1-42a0cb9528d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:13.095562 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baed06-d307-4c68-b9b1-42a0cb9528d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:13.095849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baed06-d307-4c68-b9b1-42a0cb9528d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:13.096126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baed06-d307-4c68-b9b1-42a0cb9528d6 admin admin] PATH_INFO: `/v3/projects/872dd756982b4d9585b456641b67d12b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:13.097449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20baed06-d307-4c68-b9b1-42a0cb9528d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=872dd756982b4d9585b456641b67d12b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:13.099820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20baed06-d307-4c68-b9b1-42a0cb9528d6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:13.244832 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-20baed06-d307-4c68-b9b1-42a0cb9528d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '872dd756982b4d9585b456641b67d12b', 'request_id': 'req-20baed06-d307-4c68-b9b1-42a0cb9528d6'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:13.305897 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab487f01-f2e7-4528-a3b4-0e753094ec3d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:13.307522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ab487f01-f2e7-4528-a3b4-0e753094ec3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:13.333434 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab487f01-f2e7-4528-a3b4-0e753094ec3d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:13.343240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-20baed06-d307-4c68-b9b1-42a0cb9528d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '872dd756982b4d9585b456641b67d12b', 'request_id': 'req-20baed06-d307-4c68-b9b1-42a0cb9528d6'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:13.358039 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-20baed06-d307-4c68-b9b1-42a0cb9528d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '872dd756982b4d9585b456641b67d12b', 'request_id': 'req-20baed06-d307-4c68-b9b1-42a0cb9528d6'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:13.381459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-20baed06-d307-4c68-b9b1-42a0cb9528d6 admin admin] The token cache is being invalidate because project 872dd756982b4d9585b456641b67d12b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:13.382441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-20baed06-d307-4c68-b9b1-42a0cb9528d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:13.383763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-20baed06-d307-4c68-b9b1-42a0cb9528d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:13.393259 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 963/1895] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:12 2022] DELETE /identity/v3/projects/872dd756982b4d9585b456641b67d12b => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:13.395591 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc3306-012a-4980-817c-4c24e63bdb4d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:13.399470 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc3306-012a-4980-817c-4c24e63bdb4d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:13.399470 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc3306-012a-4980-817c-4c24e63bdb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:13.433620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-8afc3306-012a-4980-817c-4c24e63bdb4d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:13.436577 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8afc3306-012a-4980-817c-4c24e63bdb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:13.516906 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830185c-dfef-4a21-9731-f59fab24a681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:13.516906 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830185c-dfef-4a21-9731-f59fab24a681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:13.516906 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830185c-dfef-4a21-9731-f59fab24a681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:13.516906 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830185c-dfef-4a21-9731-f59fab24a681 admin admin] PATH_INFO: `/v3/projects/187b9e7d610b4718bf609777d4033ff2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:13.517265 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:13.517362 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830185c-dfef-4a21-9731-f59fab24a681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=187b9e7d610b4718bf609777d4033ff2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:13.519412 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830185c-dfef-4a21-9731-f59fab24a681 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:13.534164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:13.554430 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 964/1896] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:13.562613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28300b8-f8d2-4bf4-a4ad-da6930c09a35 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:13.566364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28300b8-f8d2-4bf4-a4ad-da6930c09a35 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:13.566364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28300b8-f8d2-4bf4-a4ad-da6930c09a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:13.594549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d28300b8-f8d2-4bf4-a4ad-da6930c09a35 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:13.595906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d28300b8-f8d2-4bf4-a4ad-da6930c09a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:13.606424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 965/1897] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:13.610728 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28300b8-f8d2-4bf4-a4ad-da6930c09a35 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:13.627604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28300b8-f8d2-4bf4-a4ad-da6930c09a35 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:13.627604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f38d9d-fd2c-46b1-ab30-18ef3ba1b847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:13.628089 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f38d9d-fd2c-46b1-ab30-18ef3ba1b847 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:13.628381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f38d9d-fd2c-46b1-ab30-18ef3ba1b847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:13.628660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f38d9d-fd2c-46b1-ab30-18ef3ba1b847 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:13.629546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f38d9d-fd2c-46b1-ab30-18ef3ba1b847 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:09:13.629799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f38d9d-fd2c-46b1-ab30-18ef3ba1b847 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:13.631611 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f38d9d-fd2c-46b1-ab30-18ef3ba1b847 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:13.651234 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 966/1898] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:09:13.651234 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f38d9d-fd2c-46b1-ab30-18ef3ba1b847 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:13.651234 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-66f38d9d-fd2c-46b1-ab30-18ef3ba1b847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:13.667470 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c830185c-dfef-4a21-9731-f59fab24a681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '187b9e7d610b4718bf609777d4033ff2', 'request_id': 'req-c830185c-dfef-4a21-9731-f59fab24a681'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:13.670008 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f38d9d-fd2c-46b1-ab30-18ef3ba1b847 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:13.762051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82607027-c4ab-45b7-abcc-874e8b280833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:13.762051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82607027-c4ab-45b7-abcc-874e8b280833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:13.762401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82607027-c4ab-45b7-abcc-874e8b280833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:13.762519 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82607027-c4ab-45b7-abcc-874e8b280833 admin admin] PATH_INFO: `/v3/projects/236ebed18363417bbbd62af0c881a25b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:13.763815 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82607027-c4ab-45b7-abcc-874e8b280833 admin admin] RBAC: Authorizing `identity:delete_project(project_id=236ebed18363417bbbd62af0c881a25b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:13.765796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82607027-c4ab-45b7-abcc-874e8b280833 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:13.767816 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c830185c-dfef-4a21-9731-f59fab24a681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '187b9e7d610b4718bf609777d4033ff2', 'request_id': 'req-c830185c-dfef-4a21-9731-f59fab24a681'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:13.786800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c830185c-dfef-4a21-9731-f59fab24a681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '187b9e7d610b4718bf609777d4033ff2', 'request_id': 'req-c830185c-dfef-4a21-9731-f59fab24a681'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:13.804873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c830185c-dfef-4a21-9731-f59fab24a681 admin admin] The token cache is being invalidate because project 187b9e7d610b4718bf609777d4033ff2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:13.805108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c830185c-dfef-4a21-9731-f59fab24a681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:13.806298 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c830185c-dfef-4a21-9731-f59fab24a681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:13.807940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 933/1899] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:13 2022] DELETE /identity/v3/projects/187b9e7d610b4718bf609777d4033ff2 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:13.812729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830185c-dfef-4a21-9731-f59fab24a681 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:13.814117 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c830185c-dfef-4a21-9731-f59fab24a681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:13.829330 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830185c-dfef-4a21-9731-f59fab24a681 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:13.834839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a278d1-f880-43d2-a078-f6c6007977f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:13.835579 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a278d1-f880-43d2-a078-f6c6007977f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:13.835871 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a278d1-f880-43d2-a078-f6c6007977f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:13.836158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a278d1-f880-43d2-a078-f6c6007977f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:13.836866 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a278d1-f880-43d2-a078-f6c6007977f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:13.838801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a278d1-f880-43d2-a078-f6c6007977f2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:13.908444 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-82607027-c4ab-45b7-abcc-874e8b280833 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '236ebed18363417bbbd62af0c881a25b', 'request_id': 'req-82607027-c4ab-45b7-abcc-874e8b280833'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:13.970770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 934/1900] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:13 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:13.974274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e57996-67e2-4666-a063-83f1e985f7fd None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:13.974878 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e57996-67e2-4666-a063-83f1e985f7fd None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:13.975389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e57996-67e2-4666-a063-83f1e985f7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:13.998733 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-82607027-c4ab-45b7-abcc-874e8b280833 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '236ebed18363417bbbd62af0c881a25b', 'request_id': 'req-82607027-c4ab-45b7-abcc-874e8b280833'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:14.004039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-69e57996-67e2-4666-a063-83f1e985f7fd None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:14.005423 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-69e57996-67e2-4666-a063-83f1e985f7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:14.014458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-82607027-c4ab-45b7-abcc-874e8b280833 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '236ebed18363417bbbd62af0c881a25b', 'request_id': 'req-82607027-c4ab-45b7-abcc-874e8b280833'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:14.028895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-82607027-c4ab-45b7-abcc-874e8b280833 admin admin] The token cache is being invalidate because project 236ebed18363417bbbd62af0c881a25b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:14.029188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-82607027-c4ab-45b7-abcc-874e8b280833 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:14.029899 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-82607027-c4ab-45b7-abcc-874e8b280833 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:14.031680 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 967/1901] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:13 2022] DELETE /identity/v3/projects/236ebed18363417bbbd62af0c881a25b => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:14.035615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82607027-c4ab-45b7-abcc-874e8b280833 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:14.037017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-82607027-c4ab-45b7-abcc-874e8b280833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:14.115440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 935/1902] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:14.115440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54f039-e15e-4563-8acb-a54e8075a5e7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.115440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54f039-e15e-4563-8acb-a54e8075a5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.115440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54f039-e15e-4563-8acb-a54e8075a5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:14.129710 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82607027-c4ab-45b7-abcc-874e8b280833 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:14.137740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717400a5-6d4b-45d2-96ce-193e606eb5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:14.138765 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717400a5-6d4b-45d2-96ce-193e606eb5f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.139059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717400a5-6d4b-45d2-96ce-193e606eb5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.139397 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717400a5-6d4b-45d2-96ce-193e606eb5f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:14.139878 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-2b54f039-e15e-4563-8acb-a54e8075a5e7 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:14.141363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717400a5-6d4b-45d2-96ce-193e606eb5f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:14.141484 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2b54f039-e15e-4563-8acb-a54e8075a5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:14.142862 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717400a5-6d4b-45d2-96ce-193e606eb5f8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:14.161897 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 936/1903] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:14.161897 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ed8ea-0110-4640-875f-b48c18f43078 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.161897 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ed8ea-0110-4640-875f-b48c18f43078 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.161897 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ed8ea-0110-4640-875f-b48c18f43078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:14.184283 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-cf2ed8ea-0110-4640-875f-b48c18f43078 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:14.185526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-cf2ed8ea-0110-4640-875f-b48c18f43078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:14.195868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 937/1904] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:14.198310 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2de474-2541-430e-bd95-acab58804013 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.198814 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2de474-2541-430e-bd95-acab58804013 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.199342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2de474-2541-430e-bd95-acab58804013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:14.224014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0d2de474-2541-430e-bd95-acab58804013 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:14.225977 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0d2de474-2541-430e-bd95-acab58804013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:14.228167 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:14.236521 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 938/1905] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:14.237649 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2de474-2541-430e-bd95-acab58804013 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:14.250893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2de474-2541-430e-bd95-acab58804013 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:14.257782 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5740a1c-b498-4e38-bea7-cf732cf8fe7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:14.259033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5740a1c-b498-4e38-bea7-cf732cf8fe7c admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.259521 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5740a1c-b498-4e38-bea7-cf732cf8fe7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.259865 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5740a1c-b498-4e38-bea7-cf732cf8fe7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:14.260956 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5740a1c-b498-4e38-bea7-cf732cf8fe7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:09:14.261604 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5740a1c-b498-4e38-bea7-cf732cf8fe7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:14.264826 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5740a1c-b498-4e38-bea7-cf732cf8fe7c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:14.267706 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 968/1906] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:14 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:14.267706 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717400a5-6d4b-45d2-96ce-193e606eb5f8 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:14.283503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717400a5-6d4b-45d2-96ce-193e606eb5f8 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:14.284466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 939/1907] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:09:14.284965 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5740a1c-b498-4e38-bea7-cf732cf8fe7c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:14.293774 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94580c-582c-4ae5-9378-c28ef5129be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:14.294618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94580c-582c-4ae5-9378-c28ef5129be9 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.295267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5740a1c-b498-4e38-bea7-cf732cf8fe7c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:14.295402 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94580c-582c-4ae5-9378-c28ef5129be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.295645 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94580c-582c-4ae5-9378-c28ef5129be9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:14.296422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94580c-582c-4ae5-9378-c28ef5129be9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:09:14.296684 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94580c-582c-4ae5-9378-c28ef5129be9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:14.298305 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94580c-582c-4ae5-9378-c28ef5129be9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:14.300186 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006129f-5c64-4334-93dd-162ba6408acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:14.300938 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006129f-5c64-4334-93dd-162ba6408acf admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.301233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006129f-5c64-4334-93dd-162ba6408acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.301525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006129f-5c64-4334-93dd-162ba6408acf admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:14.302254 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006129f-5c64-4334-93dd-162ba6408acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:14.303945 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006129f-5c64-4334-93dd-162ba6408acf admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:14.316482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 969/1908] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:09:14.316482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94580c-582c-4ae5-9378-c28ef5129be9 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:14.316969 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 940/1909] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:14.317892 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006129f-5c64-4334-93dd-162ba6408acf admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:14.331615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94580c-582c-4ae5-9378-c28ef5129be9 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:14.333237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006129f-5c64-4334-93dd-162ba6408acf admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:14.338437 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369668d-1cae-4c8b-a77c-db2b1ccfc8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:14.339158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369668d-1cae-4c8b-a77c-db2b1ccfc8a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.339673 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369668d-1cae-4c8b-a77c-db2b1ccfc8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.340215 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369668d-1cae-4c8b-a77c-db2b1ccfc8a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:14.341939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369668d-1cae-4c8b-a77c-db2b1ccfc8a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:14.342887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc7d3b-6f67-47de-811c-73f12b771a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:14.343953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc7d3b-6f67-47de-811c-73f12b771a81 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.344417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc7d3b-6f67-47de-811c-73f12b771a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.344876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc7d3b-6f67-47de-811c-73f12b771a81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:14.348489 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369668d-1cae-4c8b-a77c-db2b1ccfc8a1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:14.348662 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc7d3b-6f67-47de-811c-73f12b771a81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:14.349864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc7d3b-6f67-47de-811c-73f12b771a81 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:14.459539 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 941/1910] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:14 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:14.460001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc7d3b-6f67-47de-811c-73f12b771a81 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:14.469315 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 970/1911] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:14 2022] POST /identity/v3/projects => generated 367 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:14.470994 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369668d-1cae-4c8b-a77c-db2b1ccfc8a1 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:14.482497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:14.535771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc7d3b-6f67-47de-811c-73f12b771a81 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:14.558788 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369668d-1cae-4c8b-a77c-db2b1ccfc8a1 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:14.565340 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf4b8f-a6c2-4ed2-924e-63807aef9331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:14.566126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf4b8f-a6c2-4ed2-924e-63807aef9331 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.566502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf4b8f-a6c2-4ed2-924e-63807aef9331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.566850 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf4b8f-a6c2-4ed2-924e-63807aef9331 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:14.567665 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf4b8f-a6c2-4ed2-924e-63807aef9331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:14.569733 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf4b8f-a6c2-4ed2-924e-63807aef9331 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:14.600160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7251d97-0aa4-4e5e-b9b5-4125c57fe421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:14.600952 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7251d97-0aa4-4e5e-b9b5-4125c57fe421 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.602820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7251d97-0aa4-4e5e-b9b5-4125c57fe421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.602820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7251d97-0aa4-4e5e-b9b5-4125c57fe421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:14.602820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7251d97-0aa4-4e5e-b9b5-4125c57fe421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:14.605169 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7251d97-0aa4-4e5e-b9b5-4125c57fe421 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:14.620962 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 942/1912] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:14.620962 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7251d97-0aa4-4e5e-b9b5-4125c57fe421 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:14.632087 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7251d97-0aa4-4e5e-b9b5-4125c57fe421 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:14.637785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3b9a9-bc7e-4ca5-8afc-2be0bd987ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:14.638664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3b9a9-bc7e-4ca5-8afc-2be0bd987ee8 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.640517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3b9a9-bc7e-4ca5-8afc-2be0bd987ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.640517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3b9a9-bc7e-4ca5-8afc-2be0bd987ee8 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:14.640517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3b9a9-bc7e-4ca5-8afc-2be0bd987ee8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:14.644723 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3b9a9-bc7e-4ca5-8afc-2be0bd987ee8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:14.654324 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 971/1913] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:14 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:14.666747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf4b8f-a6c2-4ed2-924e-63807aef9331 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:14.672358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf4b8f-a6c2-4ed2-924e-63807aef9331 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:14.679922 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ca9b1-8907-4bcb-8065-1256bdba53cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:14.682219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ca9b1-8907-4bcb-8065-1256bdba53cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.682590 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ca9b1-8907-4bcb-8065-1256bdba53cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.683069 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ca9b1-8907-4bcb-8065-1256bdba53cc admin admin] PATH_INFO: `/v3/projects/9275e4c95f9e4cbbae63d4faa37636d5/users/65c6dbb337d64ca4bacad73c74ec9288/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:14.712089 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ca9b1-8907-4bcb-8065-1256bdba53cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9275e4c95f9e4cbbae63d4faa37636d5, user_id=65c6dbb337d64ca4bacad73c74ec9288, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:14.713533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ca9b1-8907-4bcb-8065-1256bdba53cc admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:14.765889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 943/1914] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:14 2022] POST /identity/v3/users => generated 284 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:14.765889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3b9a9-bc7e-4ca5-8afc-2be0bd987ee8 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:14.774189 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3b9a9-bc7e-4ca5-8afc-2be0bd987ee8 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:14.779116 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c16fb-92ab-470c-a91b-7bc140fccef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:14.779807 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c16fb-92ab-470c-a91b-7bc140fccef0 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.780092 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c16fb-92ab-470c-a91b-7bc140fccef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.780396 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c16fb-92ab-470c-a91b-7bc140fccef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:14.781135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c16fb-92ab-470c-a91b-7bc140fccef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:14.782857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c16fb-92ab-470c-a91b-7bc140fccef0 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:14.792206 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 972/1915] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:14 2022] PUT /identity/v3/projects/9275e4c95f9e4cbbae63d4faa37636d5/users/65c6dbb337d64ca4bacad73c74ec9288/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:14.793234 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ca9b1-8907-4bcb-8065-1256bdba53cc admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:14.798068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 944/1916] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:14.805237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037e029-849d-4e01-92d7-f4f34796fa2f None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.805813 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037e029-849d-4e01-92d7-f4f34796fa2f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.806299 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037e029-849d-4e01-92d7-f4f34796fa2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:14.844821 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-e037e029-849d-4e01-92d7-f4f34796fa2f None None] MFA Rules not processed for user `65c6dbb337d64ca4bacad73c74ec9288`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:14.846301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e037e029-849d-4e01-92d7-f4f34796fa2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:14.870545 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ca9b1-8907-4bcb-8065-1256bdba53cc admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:14.875856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5c700-39f6-45ba-8a5a-a80a820cafc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:14.876524 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5c700-39f6-45ba-8a5a-a80a820cafc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.876811 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5c700-39f6-45ba-8a5a-a80a820cafc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.877093 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5c700-39f6-45ba-8a5a-a80a820cafc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:14.877820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5c700-39f6-45ba-8a5a-a80a820cafc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:14.879780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5c700-39f6-45ba-8a5a-a80a820cafc7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:14.882350 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:14.894071 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 973/1917] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:14.894071 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5c700-39f6-45ba-8a5a-a80a820cafc7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:14.902702 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5c700-39f6-45ba-8a5a-a80a820cafc7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:14.907706 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d368c-94ec-4c8a-b13e-50c213443a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:14.908419 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d368c-94ec-4c8a-b13e-50c213443a99 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.908703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d368c-94ec-4c8a-b13e-50c213443a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.908975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d368c-94ec-4c8a-b13e-50c213443a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:14.909687 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d368c-94ec-4c8a-b13e-50c213443a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:14.911355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d368c-94ec-4c8a-b13e-50c213443a99 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:14.938647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 974/1918] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:14.938647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d368c-94ec-4c8a-b13e-50c213443a99 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:14.949885 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 945/1919] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:14.951956 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037e029-849d-4e01-92d7-f4f34796fa2f None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:14.956741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d368c-94ec-4c8a-b13e-50c213443a99 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:14.961981 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e74b9d4-a52d-4d97-9a92-c4796ffa1641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:14.962737 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74b9d4-a52d-4d97-9a92-c4796ffa1641 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.963096 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74b9d4-a52d-4d97-9a92-c4796ffa1641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.963516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74b9d4-a52d-4d97-9a92-c4796ffa1641 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:14.964384 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e74b9d4-a52d-4d97-9a92-c4796ffa1641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:14.966415 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e74b9d4-a52d-4d97-9a92-c4796ffa1641 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:14.985074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037e029-849d-4e01-92d7-f4f34796fa2f None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:14.988877 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 975/1920] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:14.988877 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e74b9d4-a52d-4d97-9a92-c4796ffa1641 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:14.998840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70466d09-55c7-4c14-bc6e-5d42d834859d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:14.998840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70466d09-55c7-4c14-bc6e-5d42d834859d admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:14.998840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70466d09-55c7-4c14-bc6e-5d42d834859d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:14.998840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70466d09-55c7-4c14-bc6e-5d42d834859d admin admin] PATH_INFO: `/v3/projects/eeef42be20a24be2a6379560083e95f1/users/4eb6edde1b5e430e92212213de710a58/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:15.001511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e74b9d4-a52d-4d97-9a92-c4796ffa1641 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:15.010002 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c0045-090e-4552-99fc-dcaa97a12b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:15.010002 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c0045-090e-4552-99fc-dcaa97a12b53 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:15.010367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c0045-090e-4552-99fc-dcaa97a12b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:15.011354 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c0045-090e-4552-99fc-dcaa97a12b53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:15.012958 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c0045-090e-4552-99fc-dcaa97a12b53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:15.014508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c0045-090e-4552-99fc-dcaa97a12b53 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:15.035947 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70466d09-55c7-4c14-bc6e-5d42d834859d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeef42be20a24be2a6379560083e95f1, user_id=4eb6edde1b5e430e92212213de710a58, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:15.039008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70466d09-55c7-4c14-bc6e-5d42d834859d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:15.044438 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:15.103454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 976/1921] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:14 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:15.104200 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c0045-090e-4552-99fc-dcaa97a12b53 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:15.123655 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 946/1922] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:14 2022] PUT /identity/v3/projects/eeef42be20a24be2a6379560083e95f1/users/4eb6edde1b5e430e92212213de710a58/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:15.124548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70466d09-55c7-4c14-bc6e-5d42d834859d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:15.178789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c0045-090e-4552-99fc-dcaa97a12b53 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:15.212544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70466d09-55c7-4c14-bc6e-5d42d834859d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:15.218409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09937e0b-ed4b-4ca6-8d70-c9332a8e2937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:15.219094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09937e0b-ed4b-4ca6-8d70-c9332a8e2937 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:15.219412 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09937e0b-ed4b-4ca6-8d70-c9332a8e2937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:15.219692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09937e0b-ed4b-4ca6-8d70-c9332a8e2937 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:15.220432 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09937e0b-ed4b-4ca6-8d70-c9332a8e2937 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:15.223287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09937e0b-ed4b-4ca6-8d70-c9332a8e2937 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:15.250510 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdcfc6-9a00-4009-9603-bd02e27b1668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:15.251663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdcfc6-9a00-4009-9603-bd02e27b1668 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:15.251957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdcfc6-9a00-4009-9603-bd02e27b1668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:15.252246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdcfc6-9a00-4009-9603-bd02e27b1668 admin admin] PATH_INFO: `/v3/projects/443b626cd6b34a23bfeff46ca00706ad/users/2a666d83a45740ef96c003efd90cec6e/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:15.272542 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fdcfc6-9a00-4009-9603-bd02e27b1668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=443b626cd6b34a23bfeff46ca00706ad, user_id=2a666d83a45740ef96c003efd90cec6e, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:15.275918 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fdcfc6-9a00-4009-9603-bd02e27b1668 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:15.299076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 947/1923] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:15 2022] POST /identity/v3/users => generated 299 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:15.300639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30586a5-aca7-43e2-b710-19ccfda36be6 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:15.300939 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30586a5-aca7-43e2-b710-19ccfda36be6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:15.301225 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30586a5-aca7-43e2-b710-19ccfda36be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:15.319997 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d30586a5-aca7-43e2-b710-19ccfda36be6 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:15.321230 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d30586a5-aca7-43e2-b710-19ccfda36be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:15.329917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 948/1924] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:15.330850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30586a5-aca7-43e2-b710-19ccfda36be6 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:15.340760 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30586a5-aca7-43e2-b710-19ccfda36be6 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:15.345661 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a7da1-46e1-4de6-be75-6f073c46b29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:15.346364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a7da1-46e1-4de6-be75-6f073c46b29a admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:15.346631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a7da1-46e1-4de6-be75-6f073c46b29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:15.346917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a7da1-46e1-4de6-be75-6f073c46b29a admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:15.347701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a7da1-46e1-4de6-be75-6f073c46b29a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:15.349340 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a7da1-46e1-4de6-be75-6f073c46b29a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:15.366716 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 977/1925] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:15 2022] PUT /identity/v3/projects/443b626cd6b34a23bfeff46ca00706ad/users/2a666d83a45740ef96c003efd90cec6e/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:15.366716 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdcfc6-9a00-4009-9603-bd02e27b1668 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:15.366979 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 949/1926] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:15.366979 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a7da1-46e1-4de6-be75-6f073c46b29a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:15.368367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-32fdcfc6-9a00-4009-9603-bd02e27b1668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:15.383389 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a7da1-46e1-4de6-be75-6f073c46b29a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:15.419990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:15.430428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdcfc6-9a00-4009-9603-bd02e27b1668 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:15.436706 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbab806-ff2c-4446-9be8-6eee589800ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:15.436706 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbab806-ff2c-4446-9be8-6eee589800ba service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:15.436706 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbab806-ff2c-4446-9be8-6eee589800ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:15.436706 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbab806-ff2c-4446-9be8-6eee589800ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:15.447408 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dd46b-9640-4f03-9c0c-35536c11bcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:15.448317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dd46b-9640-4f03-9c0c-35536c11bcc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:15.448606 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dd46b-9640-4f03-9c0c-35536c11bcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:15.448893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dd46b-9640-4f03-9c0c-35536c11bcc5 admin admin] PATH_INFO: `/v3/projects/74212f10a690443cbcd27b9ce59dd291/users/6264174a277246d2b78ab929250ab720/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:15.465042 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557dd46b-9640-4f03-9c0c-35536c11bcc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74212f10a690443cbcd27b9ce59dd291, user_id=6264174a277246d2b78ab929250ab720, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:15.468424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557dd46b-9640-4f03-9c0c-35536c11bcc5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:15.500026 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbab806-ff2c-4446-9be8-6eee589800ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:15.501492 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbab806-ff2c-4446-9be8-6eee589800ba service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:15.510983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 978/1927] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:15.512330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf8e4b-0831-4a04-84e5-631a5219f867 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:15.512624 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf8e4b-0831-4a04-84e5-631a5219f867 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:15.512906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf8e4b-0831-4a04-84e5-631a5219f867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:15.541829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-25cf8e4b-0831-4a04-84e5-631a5219f867 None None] MFA Rules not processed for user `2a666d83a45740ef96c003efd90cec6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:15.543084 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-25cf8e4b-0831-4a04-84e5-631a5219f867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:15.547707 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 950/1928] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:15 2022] PUT /identity/v3/projects/74212f10a690443cbcd27b9ce59dd291/users/6264174a277246d2b78ab929250ab720/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:15.552320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dd46b-9640-4f03-9c0c-35536c11bcc5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:15.609509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:15.624491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dd46b-9640-4f03-9c0c-35536c11bcc5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:15.629452 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ef935-5016-4b03-8996-fb52827cc68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:15.630834 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 979/1929] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:15 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:15.631072 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ef935-5016-4b03-8996-fb52827cc68d admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:15.631652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ef935-5016-4b03-8996-fb52827cc68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:15.631922 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ef935-5016-4b03-8996-fb52827cc68d admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:15.632945 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ef935-5016-4b03-8996-fb52827cc68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:15.634644 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ef935-5016-4b03-8996-fb52827cc68d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:15.640478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf8e4b-0831-4a04-84e5-631a5219f867 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:15.648009 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 951/1930] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:09:15.653509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ef935-5016-4b03-8996-fb52827cc68d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:15.654553 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf8e4b-0831-4a04-84e5-631a5219f867 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:15.667068 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66bc847-5411-4596-b026-0a2333845d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:15.667811 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66bc847-5411-4596-b026-0a2333845d28 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:15.668074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ef935-5016-4b03-8996-fb52827cc68d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:15.668224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66bc847-5411-4596-b026-0a2333845d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:15.669010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66bc847-5411-4596-b026-0a2333845d28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:15.669635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66bc847-5411-4596-b026-0a2333845d28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:15.676141 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66bc847-5411-4596-b026-0a2333845d28 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:15.677706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5dea5-c6db-4b3f-afc9-a8d6f5a9111a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:15.679855 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5dea5-c6db-4b3f-afc9-a8d6f5a9111a admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:15.680332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5dea5-c6db-4b3f-afc9-a8d6f5a9111a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:15.683815 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5dea5-c6db-4b3f-afc9-a8d6f5a9111a admin admin] PATH_INFO: `/v3/projects/74212f10a690443cbcd27b9ce59dd291/users/6264174a277246d2b78ab929250ab720/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:15.685583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e5dea5-c6db-4b3f-afc9-a8d6f5a9111a admin admin] RBAC: Authorizing `identity:create_grant(project_id=74212f10a690443cbcd27b9ce59dd291, user_id=6264174a277246d2b78ab929250ab720, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:15.690580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e5dea5-c6db-4b3f-afc9-a8d6f5a9111a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:15.713864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 952/1931] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:15 2022] PUT /identity/v3/projects/74212f10a690443cbcd27b9ce59dd291/users/6264174a277246d2b78ab929250ab720/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:15.729028 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141024fd-0474-449f-ae36-273add8889d5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:15.729372 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141024fd-0474-449f-ae36-273add8889d5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:15.729710 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141024fd-0474-449f-ae36-273add8889d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:15.766322 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-141024fd-0474-449f-ae36-273add8889d5 None None] MFA Rules not processed for user `6264174a277246d2b78ab929250ab720`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:15.768360 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-141024fd-0474-449f-ae36-273add8889d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:15.815340 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 980/1932] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:15 2022] POST /identity/v3/projects => generated 369 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:09:15.817719 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66bc847-5411-4596-b026-0a2333845d28 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:15.971862 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 953/1933] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:09:15.983957 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141024fd-0474-449f-ae36-273add8889d5 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:15.986363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66bc847-5411-4596-b026-0a2333845d28 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:16.003329 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-141024fd-0474-449f-ae36-273add8889d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:16.005983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4223117-a90f-4a20-9ed9-4160603dab1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:16.008352 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4223117-a90f-4a20-9ed9-4160603dab1e admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:16.010607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4223117-a90f-4a20-9ed9-4160603dab1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:16.011393 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4223117-a90f-4a20-9ed9-4160603dab1e admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:16.017855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4223117-a90f-4a20-9ed9-4160603dab1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:16.021133 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4223117-a90f-4a20-9ed9-4160603dab1e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:16.034506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:16.085740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141024fd-0474-449f-ae36-273add8889d5 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:16.143446 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 981/1934] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:15 2022] POST /identity/v3/users => generated 291 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:16.144354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f367616-e525-461d-8496-566c5a232915 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:16.145027 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f367616-e525-461d-8496-566c5a232915 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:16.147970 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f367616-e525-461d-8496-566c5a232915 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:16.147970 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f367616-e525-461d-8496-566c5a232915 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:16.148289 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4223117-a90f-4a20-9ed9-4160603dab1e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:16.159272 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4223117-a90f-4a20-9ed9-4160603dab1e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:16.164436 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc6a76-5d46-470c-974a-66f3727e96ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:16.165156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc6a76-5d46-470c-974a-66f3727e96ca admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:16.165439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc6a76-5d46-470c-974a-66f3727e96ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:16.165719 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc6a76-5d46-470c-974a-66f3727e96ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:16.166530 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc6a76-5d46-470c-974a-66f3727e96ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:16.172927 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc6a76-5d46-470c-974a-66f3727e96ca admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:16.177664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:16.191223 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 982/1935] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:09:16.195318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc6a76-5d46-470c-974a-66f3727e96ca admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:16.206253 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc6a76-5d46-470c-974a-66f3727e96ca admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:16.213131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969335b5-75dd-47ae-84df-41395b0bb6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:16.214128 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969335b5-75dd-47ae-84df-41395b0bb6a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:16.214576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969335b5-75dd-47ae-84df-41395b0bb6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:16.214967 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969335b5-75dd-47ae-84df-41395b0bb6a6 admin admin] PATH_INFO: `/v3/projects/b6f25f7861584475903616c93bdced84/users/67742c4e9d464f5cb76f2396c4316305/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:16.235607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969335b5-75dd-47ae-84df-41395b0bb6a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f25f7861584475903616c93bdced84, user_id=67742c4e9d464f5cb76f2396c4316305, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:16.238836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969335b5-75dd-47ae-84df-41395b0bb6a6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:16.239535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f367616-e525-461d-8496-566c5a232915 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:16.242028 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f367616-e525-461d-8496-566c5a232915 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:16.255880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 954/1936] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:16.286876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f367616-e525-461d-8496-566c5a232915 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:16.300696 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f367616-e525-461d-8496-566c5a232915 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:16.309922 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582334e-3a50-4752-bb78-d081f046177c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:16.310859 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582334e-3a50-4752-bb78-d081f046177c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:16.311151 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582334e-3a50-4752-bb78-d081f046177c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:16.311464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582334e-3a50-4752-bb78-d081f046177c admin admin] PATH_INFO: `/v3/users/65c6dbb337d64ca4bacad73c74ec9288` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:16.312695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3582334e-3a50-4752-bb78-d081f046177c admin admin] RBAC: Authorizing `identity:delete_user(user_id=65c6dbb337d64ca4bacad73c74ec9288)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:16.314881 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3582334e-3a50-4752-bb78-d081f046177c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:16.359336 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 983/1937] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:16 2022] PUT /identity/v3/projects/b6f25f7861584475903616c93bdced84/users/67742c4e9d464f5cb76f2396c4316305/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:16.363590 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969335b5-75dd-47ae-84df-41395b0bb6a6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:16.434180 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969335b5-75dd-47ae-84df-41395b0bb6a6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:16.438985 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70276d00-0a99-445d-bdac-9d7b9cea0447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:16.439699 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70276d00-0a99-445d-bdac-9d7b9cea0447 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:16.440180 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70276d00-0a99-445d-bdac-9d7b9cea0447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:16.440685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70276d00-0a99-445d-bdac-9d7b9cea0447 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:16.441438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70276d00-0a99-445d-bdac-9d7b9cea0447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:16.443082 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70276d00-0a99-445d-bdac-9d7b9cea0447 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:16.455434 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 984/1938] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:09:16.459518 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70276d00-0a99-445d-bdac-9d7b9cea0447 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:16.469899 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70276d00-0a99-445d-bdac-9d7b9cea0447 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:16.474626 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636d0d0-b463-4948-94a6-ba10c6cbdc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:16.475553 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636d0d0-b463-4948-94a6-ba10c6cbdc2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:16.475845 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636d0d0-b463-4948-94a6-ba10c6cbdc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:16.476122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636d0d0-b463-4948-94a6-ba10c6cbdc2d admin admin] PATH_INFO: `/v3/projects/b6f25f7861584475903616c93bdced84/users/67742c4e9d464f5cb76f2396c4316305/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:16.478179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636d0d0-b463-4948-94a6-ba10c6cbdc2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f25f7861584475903616c93bdced84, user_id=67742c4e9d464f5cb76f2396c4316305, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:16.480940 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636d0d0-b463-4948-94a6-ba10c6cbdc2d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:16.494776 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 985/1939] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:16 2022] PUT /identity/v3/projects/b6f25f7861584475903616c93bdced84/users/67742c4e9d464f5cb76f2396c4316305/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:16.506293 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20759799-e8cd-499f-b2f4-ffc069f6c684 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:16.507388 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20759799-e8cd-499f-b2f4-ffc069f6c684 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:16.507650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20759799-e8cd-499f-b2f4-ffc069f6c684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:16.541146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-20759799-e8cd-499f-b2f4-ffc069f6c684 None None] MFA Rules not processed for user `67742c4e9d464f5cb76f2396c4316305`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:16.542397 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-20759799-e8cd-499f-b2f4-ffc069f6c684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:16.585560 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3582334e-3a50-4752-bb78-d081f046177c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65c6dbb337d64ca4bacad73c74ec9288'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:16.594543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3582334e-3a50-4752-bb78-d081f046177c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65c6dbb337d64ca4bacad73c74ec9288'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:16.595460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3582334e-3a50-4752-bb78-d081f046177c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65c6dbb337d64ca4bacad73c74ec9288'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:16.596266 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3582334e-3a50-4752-bb78-d081f046177c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65c6dbb337d64ca4bacad73c74ec9288'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:16.610076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3582334e-3a50-4752-bb78-d081f046177c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65c6dbb337d64ca4bacad73c74ec9288'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:16.644946 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 986/1940] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:16 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:09:16.657139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20759799-e8cd-499f-b2f4-ffc069f6c684 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:16.659137 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-20759799-e8cd-499f-b2f4-ffc069f6c684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:16.677618 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 955/1941] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:16 2022] DELETE /identity/v3/users/65c6dbb337d64ca4bacad73c74ec9288 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:16.683287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ede2e2-d720-466d-8c60-2d2b629f5668 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:16.684065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ede2e2-d720-466d-8c60-2d2b629f5668 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:16.684605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ede2e2-d720-466d-8c60-2d2b629f5668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:16.695645 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:16.709871 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b9ede2e2-d720-466d-8c60-2d2b629f5668 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:16.712095 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b9ede2e2-d720-466d-8c60-2d2b629f5668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:16.759047 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20759799-e8cd-499f-b2f4-ffc069f6c684 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:16.817773 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 956/1942] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:16.825467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ede2e2-d720-466d-8c60-2d2b629f5668 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:16.828503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b9ede2e2-d720-466d-8c60-2d2b629f5668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:16.876832 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f79248-70f3-477f-ac1e-f7530cb8510d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:16.877785 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f79248-70f3-477f-ac1e-f7530cb8510d service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:16.878133 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f79248-70f3-477f-ac1e-f7530cb8510d service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:16.878476 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f79248-70f3-477f-ac1e-f7530cb8510d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:16.899251 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:16.921466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ede2e2-d720-466d-8c60-2d2b629f5668 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:16.927475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472af5e4-2dd8-4bf3-89e4-7b97dca33bdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:16.928873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472af5e4-2dd8-4bf3-89e4-7b97dca33bdb service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:16.929407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472af5e4-2dd8-4bf3-89e4-7b97dca33bdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:16.929898 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472af5e4-2dd8-4bf3-89e4-7b97dca33bdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:16.943040 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472af5e4-2dd8-4bf3-89e4-7b97dca33bdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:16.944448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472af5e4-2dd8-4bf3-89e4-7b97dca33bdb service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:16.955383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 957/1943] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:16.961461 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f79248-70f3-477f-ac1e-f7530cb8510d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:16.963898 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f79248-70f3-477f-ac1e-f7530cb8510d service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:16.974936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 987/1944] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:16 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:09:17.164005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43e666-89a1-4921-befa-45ff2fe5c1a1 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:17.164005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43e666-89a1-4921-befa-45ff2fe5c1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:17.164626 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43e666-89a1-4921-befa-45ff2fe5c1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:17.252673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-5c43e666-89a1-4921-befa-45ff2fe5c1a1 None None] MFA Rules not processed for user `4eb6edde1b5e430e92212213de710a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:17.254063 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5c43e666-89a1-4921-befa-45ff2fe5c1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:17.392440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 958/1945] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:09:17.396271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f79248-70f3-477f-ac1e-f7530cb8510d service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:17.398484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e8f79248-70f3-477f-ac1e-f7530cb8510d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:17.438426 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f79248-70f3-477f-ac1e-f7530cb8510d service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:17.452271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53f1d5-10b4-4838-8aea-abfb279f89cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:17.453169 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53f1d5-10b4-4838-8aea-abfb279f89cf admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:17.453632 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53f1d5-10b4-4838-8aea-abfb279f89cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:17.454062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53f1d5-10b4-4838-8aea-abfb279f89cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:17.454961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53f1d5-10b4-4838-8aea-abfb279f89cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:17.458808 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53f1d5-10b4-4838-8aea-abfb279f89cf admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:17.575975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 988/1946] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:17 2022] POST /identity/v3/projects => generated 371 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:17.577495 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43e666-89a1-4921-befa-45ff2fe5c1a1 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:17.598206 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43e666-89a1-4921-befa-45ff2fe5c1a1 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:17.669529 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959929c-2c0b-4e79-94f8-b72ab9932060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:17.670771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959929c-2c0b-4e79-94f8-b72ab9932060 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:17.671252 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959929c-2c0b-4e79-94f8-b72ab9932060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:17.671751 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959929c-2c0b-4e79-94f8-b72ab9932060 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:17.673141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959929c-2c0b-4e79-94f8-b72ab9932060 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:17.676053 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53f1d5-10b4-4838-8aea-abfb279f89cf admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:17.677023 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959929c-2c0b-4e79-94f8-b72ab9932060 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:17.678372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bf53f1d5-10b4-4838-8aea-abfb279f89cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:17.693213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53f1d5-10b4-4838-8aea-abfb279f89cf admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:17.706601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48a688-30a9-42c6-be47-6f981f9615f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:17.707802 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48a688-30a9-42c6-be47-6f981f9615f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:17.708348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48a688-30a9-42c6-be47-6f981f9615f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:17.708828 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48a688-30a9-42c6-be47-6f981f9615f6 admin admin] PATH_INFO: `/v3/projects/9275e4c95f9e4cbbae63d4faa37636d5` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:17.712088 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48a688-30a9-42c6-be47-6f981f9615f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9275e4c95f9e4cbbae63d4faa37636d5)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:17.716278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48a688-30a9-42c6-be47-6f981f9615f6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:17.790247 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 959/1947] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:17 2022] POST /identity/v3/users => generated 292 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:09:17.795324 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959929c-2c0b-4e79-94f8-b72ab9932060 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:17.805690 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959929c-2c0b-4e79-94f8-b72ab9932060 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:17.811012 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f99ef-4f58-4e94-a90b-257029e545df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:17.811801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f99ef-4f58-4e94-a90b-257029e545df admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:17.812099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f99ef-4f58-4e94-a90b-257029e545df admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:17.812380 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f99ef-4f58-4e94-a90b-257029e545df admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:17.813112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f99ef-4f58-4e94-a90b-257029e545df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:17.814732 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f99ef-4f58-4e94-a90b-257029e545df admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:17.827236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 960/1948] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:09:17.829535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f99ef-4f58-4e94-a90b-257029e545df admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:17.839567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f99ef-4f58-4e94-a90b-257029e545df admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:17.847820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402b136-ceb7-4761-a247-6a19b6e58e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:17.848722 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402b136-ceb7-4761-a247-6a19b6e58e52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:17.849039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402b136-ceb7-4761-a247-6a19b6e58e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:17.849322 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402b136-ceb7-4761-a247-6a19b6e58e52 admin admin] PATH_INFO: `/v3/projects/09814c09a8594e62b20c5581c1eae8df/users/0893b04c88d84c378522f4531e12cf9e/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:17.885089 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1402b136-ceb7-4761-a247-6a19b6e58e52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09814c09a8594e62b20c5581c1eae8df, user_id=0893b04c88d84c378522f4531e12cf9e, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:17.886277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1402b136-ceb7-4761-a247-6a19b6e58e52 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:17.899451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8b48a688-30a9-42c6-be47-6f981f9615f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9275e4c95f9e4cbbae63d4faa37636d5', 'request_id': 'req-8b48a688-30a9-42c6-be47-6f981f9615f6'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:18.007886 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8b48a688-30a9-42c6-be47-6f981f9615f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9275e4c95f9e4cbbae63d4faa37636d5', 'request_id': 'req-8b48a688-30a9-42c6-be47-6f981f9615f6'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:18.016199 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:18.016708 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 961/1949] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:17 2022] PUT /identity/v3/projects/09814c09a8594e62b20c5581c1eae8df/users/0893b04c88d84c378522f4531e12cf9e/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:18.018582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8b48a688-30a9-42c6-be47-6f981f9615f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9275e4c95f9e4cbbae63d4faa37636d5', 'request_id': 'req-8b48a688-30a9-42c6-be47-6f981f9615f6'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:18.020806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402b136-ceb7-4761-a247-6a19b6e58e52 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:18.037240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8b48a688-30a9-42c6-be47-6f981f9615f6 admin admin] The token cache is being invalidate because project 9275e4c95f9e4cbbae63d4faa37636d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:18.037532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8b48a688-30a9-42c6-be47-6f981f9615f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:18.038217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8b48a688-30a9-42c6-be47-6f981f9615f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:18.040083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 989/1950] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:17 2022] DELETE /identity/v3/projects/9275e4c95f9e4cbbae63d4faa37636d5 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:18.047910 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48a688-30a9-42c6-be47-6f981f9615f6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:18.047910 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8b48a688-30a9-42c6-be47-6f981f9615f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:18.062824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:18.106948 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402b136-ceb7-4761-a247-6a19b6e58e52 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:18.111677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc236a-6fae-4742-ae17-0bb812b7d704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:18.112362 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc236a-6fae-4742-ae17-0bb812b7d704 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:18.112650 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc236a-6fae-4742-ae17-0bb812b7d704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:18.112931 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc236a-6fae-4742-ae17-0bb812b7d704 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:18.113664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc236a-6fae-4742-ae17-0bb812b7d704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:18.115319 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc236a-6fae-4742-ae17-0bb812b7d704 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:18.125719 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 962/1951] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:09:18.159320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc236a-6fae-4742-ae17-0bb812b7d704 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:18.159320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a2fc236a-6fae-4742-ae17-0bb812b7d704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:18.159320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc236a-6fae-4742-ae17-0bb812b7d704 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:18.161495 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48a688-30a9-42c6-be47-6f981f9615f6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:18.162037 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11178ad-ec6c-4fde-aaa8-3d1c1642f87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:18.162220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23838f42-8143-4a49-8b83-2fe6c68345e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:18.162313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11178ad-ec6c-4fde-aaa8-3d1c1642f87a admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:18.162313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11178ad-ec6c-4fde-aaa8-3d1c1642f87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:18.162438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23838f42-8143-4a49-8b83-2fe6c68345e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:18.162517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11178ad-ec6c-4fde-aaa8-3d1c1642f87a admin admin] PATH_INFO: `/v3/projects/09814c09a8594e62b20c5581c1eae8df/users/0893b04c88d84c378522f4531e12cf9e/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:18.162597 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23838f42-8143-4a49-8b83-2fe6c68345e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:18.162597 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23838f42-8143-4a49-8b83-2fe6c68345e1 admin admin] PATH_INFO: `/v3/users/6264174a277246d2b78ab929250ab720` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:18.162597 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23838f42-8143-4a49-8b83-2fe6c68345e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6264174a277246d2b78ab929250ab720)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:18.162772 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11178ad-ec6c-4fde-aaa8-3d1c1642f87a admin admin] RBAC: Authorizing `identity:create_grant(project_id=09814c09a8594e62b20c5581c1eae8df, user_id=0893b04c88d84c378522f4531e12cf9e, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:18.162850 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23838f42-8143-4a49-8b83-2fe6c68345e1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:18.162920 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11178ad-ec6c-4fde-aaa8-3d1c1642f87a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:18.173175 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 963/1952] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:18 2022] PUT /identity/v3/projects/09814c09a8594e62b20c5581c1eae8df/users/0893b04c88d84c378522f4531e12cf9e/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:18.435895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-23838f42-8143-4a49-8b83-2fe6c68345e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6264174a277246d2b78ab929250ab720'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:18.436673 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-23838f42-8143-4a49-8b83-2fe6c68345e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6264174a277246d2b78ab929250ab720'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:18.451841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-23838f42-8143-4a49-8b83-2fe6c68345e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6264174a277246d2b78ab929250ab720'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:18.452588 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-23838f42-8143-4a49-8b83-2fe6c68345e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6264174a277246d2b78ab929250ab720'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:18.459631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-23838f42-8143-4a49-8b83-2fe6c68345e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6264174a277246d2b78ab929250ab720'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:18.576028 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 990/1953] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:18 2022] DELETE /identity/v3/users/6264174a277246d2b78ab929250ab720 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:19.240669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11178ad-ec6c-4fde-aaa8-3d1c1642f87a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:19.243090 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d11178ad-ec6c-4fde-aaa8-3d1c1642f87a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:19.320753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11178ad-ec6c-4fde-aaa8-3d1c1642f87a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:19.394192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9fa09-2531-4346-958a-bb3e236532c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:19.395679 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9fa09-2531-4346-958a-bb3e236532c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:19.398182 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9fa09-2531-4346-958a-bb3e236532c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:19.405462 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9fa09-2531-4346-958a-bb3e236532c2 admin admin] PATH_INFO: `/v3/projects/74212f10a690443cbcd27b9ce59dd291` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:19.405462 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9fa09-2531-4346-958a-bb3e236532c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74212f10a690443cbcd27b9ce59dd291)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:19.405462 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9fa09-2531-4346-958a-bb3e236532c2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:19.544499 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-31a9fa09-2531-4346-958a-bb3e236532c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74212f10a690443cbcd27b9ce59dd291', 'request_id': 'req-31a9fa09-2531-4346-958a-bb3e236532c2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:19.550239 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:19.647526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-31a9fa09-2531-4346-958a-bb3e236532c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74212f10a690443cbcd27b9ce59dd291', 'request_id': 'req-31a9fa09-2531-4346-958a-bb3e236532c2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:19.654726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-31a9fa09-2531-4346-958a-bb3e236532c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74212f10a690443cbcd27b9ce59dd291', 'request_id': 'req-31a9fa09-2531-4346-958a-bb3e236532c2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:19.670248 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-31a9fa09-2531-4346-958a-bb3e236532c2 admin admin] The token cache is being invalidate because project 74212f10a690443cbcd27b9ce59dd291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:19.670597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-31a9fa09-2531-4346-958a-bb3e236532c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:19.671272 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-31a9fa09-2531-4346-958a-bb3e236532c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:19.672834 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 964/1954] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:19 2022] DELETE /identity/v3/projects/74212f10a690443cbcd27b9ce59dd291 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:19.678188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fb3bb-ce22-4d3d-a4ff-4fb6fa30fd8b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:19.679016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fb3bb-ce22-4d3d-a4ff-4fb6fa30fd8b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:19.679016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fb3bb-ce22-4d3d-a4ff-4fb6fa30fd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:19.707616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ca2fb3bb-ce22-4d3d-a4ff-4fb6fa30fd8b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:19.709295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ca2fb3bb-ce22-4d3d-a4ff-4fb6fa30fd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:19.813866 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ff21b-b5c8-4dfe-93b3-32bc0565e170 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:19.814166 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ff21b-b5c8-4dfe-93b3-32bc0565e170 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:19.814501 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ff21b-b5c8-4dfe-93b3-32bc0565e170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:19.824606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 991/1955] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:19.835463 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb8654-ffdf-4ab1-8e8e-647e88fd48d5 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:19.836515 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb8654-ffdf-4ab1-8e8e-647e88fd48d5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:19.837496 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb8654-ffdf-4ab1-8e8e-647e88fd48d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:19.869194 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e7fb8654-ffdf-4ab1-8e8e-647e88fd48d5 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:19.871264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e7fb8654-ffdf-4ab1-8e8e-647e88fd48d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:19.883793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ff1ff21b-b5c8-4dfe-93b3-32bc0565e170 None None] MFA Rules not processed for user `0893b04c88d84c378522f4531e12cf9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:19.886631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ff1ff21b-b5c8-4dfe-93b3-32bc0565e170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:19.919291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 992/1956] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:19.927823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb8654-ffdf-4ab1-8e8e-647e88fd48d5 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:19.959695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:19.984334 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb8654-ffdf-4ab1-8e8e-647e88fd48d5 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:20.010309 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51839d-7fa1-4f86-998c-f89dc12ba555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:20.017774 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51839d-7fa1-4f86-998c-f89dc12ba555 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:20.017774 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51839d-7fa1-4f86-998c-f89dc12ba555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:20.018155 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51839d-7fa1-4f86-998c-f89dc12ba555 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:20.020316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51839d-7fa1-4f86-998c-f89dc12ba555 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:09:20.021742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51839d-7fa1-4f86-998c-f89dc12ba555 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:20.035338 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51839d-7fa1-4f86-998c-f89dc12ba555 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:20.096156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 993/1957] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 19:09:20.102277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51839d-7fa1-4f86-998c-f89dc12ba555 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:20.144758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51839d-7fa1-4f86-998c-f89dc12ba555 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:20.150725 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4432f-804b-4c7c-9b7c-765a790afe48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:20.151933 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4432f-804b-4c7c-9b7c-765a790afe48 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:20.152429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4432f-804b-4c7c-9b7c-765a790afe48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:20.153049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4432f-804b-4c7c-9b7c-765a790afe48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:20.155508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4432f-804b-4c7c-9b7c-765a790afe48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:20.161992 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4432f-804b-4c7c-9b7c-765a790afe48 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:20.176526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 965/1958] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:19 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:09:20.190459 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ff21b-b5c8-4dfe-93b3-32bc0565e170 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:20.194575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ff1ff21b-b5c8-4dfe-93b3-32bc0565e170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:20.294842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ff21b-b5c8-4dfe-93b3-32bc0565e170 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:20.299544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ffca4-8bb0-4fcf-8271-99ea2f3232c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:20.300320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ffca4-8bb0-4fcf-8271-99ea2f3232c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:20.300607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ffca4-8bb0-4fcf-8271-99ea2f3232c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:20.300933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ffca4-8bb0-4fcf-8271-99ea2f3232c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:20.302894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ff7ffca4-8bb0-4fcf-8271-99ea2f3232c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:20.321446 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 994/1959] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:20 2022] POST /identity/v3/projects => generated 379 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:20.323204 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4432f-804b-4c7c-9b7c-765a790afe48 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:20.395287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ffca4-8bb0-4fcf-8271-99ea2f3232c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:20.398563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ffca4-8bb0-4fcf-8271-99ea2f3232c9 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:20.414719 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 966/1960] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:20.445193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4432f-804b-4c7c-9b7c-765a790afe48 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:20.451338 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0ddf4-2461-44a3-b12a-3f7e7b79a673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:20.452097 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0ddf4-2461-44a3-b12a-3f7e7b79a673 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:20.453079 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0ddf4-2461-44a3-b12a-3f7e7b79a673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:20.453494 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0ddf4-2461-44a3-b12a-3f7e7b79a673 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:20.454331 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0ddf4-2461-44a3-b12a-3f7e7b79a673 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:20.456706 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0ddf4-2461-44a3-b12a-3f7e7b79a673 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:20.559380 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 995/1961] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:20 2022] POST /identity/v3/users => generated 290 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:09:20.564815 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ffca4-8bb0-4fcf-8271-99ea2f3232c9 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:20.579128 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ffca4-8bb0-4fcf-8271-99ea2f3232c9 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:20.584110 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491ab3e-1f23-45a2-8901-fe57b320a470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:20.585359 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491ab3e-1f23-45a2-8901-fe57b320a470 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:20.585359 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491ab3e-1f23-45a2-8901-fe57b320a470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:20.585359 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491ab3e-1f23-45a2-8901-fe57b320a470 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:20.586082 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1491ab3e-1f23-45a2-8901-fe57b320a470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:20.587833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1491ab3e-1f23-45a2-8901-fe57b320a470 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:20.613212 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 967/1962] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:09:20.615546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0ddf4-2461-44a3-b12a-3f7e7b79a673 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:20.628418 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0ddf4-2461-44a3-b12a-3f7e7b79a673 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:20.635040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542adddf-3f31-4dfc-a0fa-84a47fb4f3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:20.636160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542adddf-3f31-4dfc-a0fa-84a47fb4f3ce admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:20.636611 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542adddf-3f31-4dfc-a0fa-84a47fb4f3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:20.637591 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542adddf-3f31-4dfc-a0fa-84a47fb4f3ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:20.638559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542adddf-3f31-4dfc-a0fa-84a47fb4f3ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:20.640776 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542adddf-3f31-4dfc-a0fa-84a47fb4f3ce admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:20.654771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 996/1963] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:20.660255 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491ab3e-1f23-45a2-8901-fe57b320a470 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:20.673360 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491ab3e-1f23-45a2-8901-fe57b320a470 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:20.679460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb0d44-2657-4cdf-a0f6-bb9c54f9dbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:20.680659 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb0d44-2657-4cdf-a0f6-bb9c54f9dbeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:20.681022 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb0d44-2657-4cdf-a0f6-bb9c54f9dbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:20.681382 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb0d44-2657-4cdf-a0f6-bb9c54f9dbeb admin admin] PATH_INFO: `/v3/projects/04e5d96927b440c1b26aeb871cc8e86b/users/3fbf4ed834704615b1ba01111f880b91/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:20.693720 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:20.703402 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfb0d44-2657-4cdf-a0f6-bb9c54f9dbeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e5d96927b440c1b26aeb871cc8e86b, user_id=3fbf4ed834704615b1ba01111f880b91, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:20.711402 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfb0d44-2657-4cdf-a0f6-bb9c54f9dbeb admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:20.805035 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 968/1964] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:20 2022] PUT /identity/v3/projects/04e5d96927b440c1b26aeb871cc8e86b/users/3fbf4ed834704615b1ba01111f880b91/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:20.816181 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e573d-4839-465e-8d33-789025fc3503 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:20.816758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e573d-4839-465e-8d33-789025fc3503 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:20.816758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e573d-4839-465e-8d33-789025fc3503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:20.848280 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ae6e573d-4839-465e-8d33-789025fc3503 None None] MFA Rules not processed for user `3fbf4ed834704615b1ba01111f880b91`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:20.849787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ae6e573d-4839-465e-8d33-789025fc3503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:20.915725 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb0d44-2657-4cdf-a0f6-bb9c54f9dbeb admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:20.918728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7bfb0d44-2657-4cdf-a0f6-bb9c54f9dbeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:20.956586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 997/1965] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:09:20.958512 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e573d-4839-465e-8d33-789025fc3503 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:21.024070 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:21.039987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb0d44-2657-4cdf-a0f6-bb9c54f9dbeb admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:21.046450 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5d8dc-3201-490b-a2c8-90d478f6e6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:21.048046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5d8dc-3201-490b-a2c8-90d478f6e6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:21.051910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5d8dc-3201-490b-a2c8-90d478f6e6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:21.051910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5d8dc-3201-490b-a2c8-90d478f6e6c0 admin admin] PATH_INFO: `/v3/users/2a666d83a45740ef96c003efd90cec6e` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:21.051910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5d8dc-3201-490b-a2c8-90d478f6e6c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a666d83a45740ef96c003efd90cec6e)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:21.053698 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5d8dc-3201-490b-a2c8-90d478f6e6c0 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:21.059394 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e573d-4839-465e-8d33-789025fc3503 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:21.065753 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b7a7f-193d-4e6e-b5d7-e8bd873105e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:21.066820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b7a7f-193d-4e6e-b5d7-e8bd873105e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:21.067375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b7a7f-193d-4e6e-b5d7-e8bd873105e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:21.083941 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b7a7f-193d-4e6e-b5d7-e8bd873105e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:21.083941 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b7a7f-193d-4e6e-b5d7-e8bd873105e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:21.083941 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b7a7f-193d-4e6e-b5d7-e8bd873105e3 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:21.238107 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 998/1966] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:20 2022] POST /identity/v3/projects => generated 383 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:21.241786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b7a7f-193d-4e6e-b5d7-e8bd873105e3 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:21.334990 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8cf5d8dc-3201-490b-a2c8-90d478f6e6c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a666d83a45740ef96c003efd90cec6e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:21.346132 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8cf5d8dc-3201-490b-a2c8-90d478f6e6c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a666d83a45740ef96c003efd90cec6e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:21.347261 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8cf5d8dc-3201-490b-a2c8-90d478f6e6c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a666d83a45740ef96c003efd90cec6e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:21.347910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8cf5d8dc-3201-490b-a2c8-90d478f6e6c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a666d83a45740ef96c003efd90cec6e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:21.367841 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8cf5d8dc-3201-490b-a2c8-90d478f6e6c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a666d83a45740ef96c003efd90cec6e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:21.368346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b7a7f-193d-4e6e-b5d7-e8bd873105e3 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:21.383850 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dca5c0-5472-48df-b934-406b127e7df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:21.386138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dca5c0-5472-48df-b934-406b127e7df6 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:21.386138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dca5c0-5472-48df-b934-406b127e7df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:21.386138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dca5c0-5472-48df-b934-406b127e7df6 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:21.386517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dca5c0-5472-48df-b934-406b127e7df6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:21.389167 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dca5c0-5472-48df-b934-406b127e7df6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:21.443798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 969/1967] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:20 2022] DELETE /identity/v3/users/2a666d83a45740ef96c003efd90cec6e => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:21.452292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945f68a-c0d8-4485-b5c9-7b5b307a73fe None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:21.453141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945f68a-c0d8-4485-b5c9-7b5b307a73fe None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:21.453593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945f68a-c0d8-4485-b5c9-7b5b307a73fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:21.485322 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d945f68a-c0d8-4485-b5c9-7b5b307a73fe None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:21.487501 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d945f68a-c0d8-4485-b5c9-7b5b307a73fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:21.491919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 999/1968] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:21 2022] POST /identity/v3/users => generated 298 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:21.493526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dca5c0-5472-48df-b934-406b127e7df6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:21.495978 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c9dca5c0-5472-48df-b934-406b127e7df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:21.519431 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dca5c0-5472-48df-b934-406b127e7df6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:21.576852 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 970/1969] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:09:21.586631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945f68a-c0d8-4485-b5c9-7b5b307a73fe None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:21.588268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d945f68a-c0d8-4485-b5c9-7b5b307a73fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:21.595833 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f74f04-3550-4138-9ce7-d7410f21ebef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:21.596526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f74f04-3550-4138-9ce7-d7410f21ebef admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:21.596834 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f74f04-3550-4138-9ce7-d7410f21ebef admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:21.597125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f74f04-3550-4138-9ce7-d7410f21ebef admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:21.597820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f74f04-3550-4138-9ce7-d7410f21ebef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:21.598105 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:21.599462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f74f04-3550-4138-9ce7-d7410f21ebef admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:21.610305 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1000/1970] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:21.614628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f74f04-3550-4138-9ce7-d7410f21ebef admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:21.625031 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f74f04-3550-4138-9ce7-d7410f21ebef admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:21.629800 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65110fd1-a5e2-4813-881e-faf6f3c1d95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:21.630703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65110fd1-a5e2-4813-881e-faf6f3c1d95d admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:21.631038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65110fd1-a5e2-4813-881e-faf6f3c1d95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:21.631287 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65110fd1-a5e2-4813-881e-faf6f3c1d95d admin admin] PATH_INFO: `/v3/projects/117523bc87394e1e80842aac0ace8280/users/018af68e9f8543adbc15b66adb788293/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:21.647857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65110fd1-a5e2-4813-881e-faf6f3c1d95d admin admin] RBAC: Authorizing `identity:create_grant(project_id=117523bc87394e1e80842aac0ace8280, user_id=018af68e9f8543adbc15b66adb788293, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:21.650527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65110fd1-a5e2-4813-881e-faf6f3c1d95d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:21.650905 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945f68a-c0d8-4485-b5c9-7b5b307a73fe None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:21.654561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436bcc4-0c3f-4e9e-9834-fcfaf43515be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:21.655245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436bcc4-0c3f-4e9e-9834-fcfaf43515be service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:21.655536 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436bcc4-0c3f-4e9e-9834-fcfaf43515be service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:21.655814 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436bcc4-0c3f-4e9e-9834-fcfaf43515be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:21.665198 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436bcc4-0c3f-4e9e-9834-fcfaf43515be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:21.666627 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436bcc4-0c3f-4e9e-9834-fcfaf43515be service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:21.676676 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 971/1971] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:09:21.734893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1001/1972] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:21 2022] PUT /identity/v3/projects/117523bc87394e1e80842aac0ace8280/users/018af68e9f8543adbc15b66adb788293/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:21.739114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436bcc4-0c3f-4e9e-9834-fcfaf43515be service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:21.749883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436bcc4-0c3f-4e9e-9834-fcfaf43515be service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:21.826081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb2c7e-7d54-47d2-9ca4-3d574f61f941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:21.826785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb2c7e-7d54-47d2-9ca4-3d574f61f941 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:21.827081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb2c7e-7d54-47d2-9ca4-3d574f61f941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:21.827401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb2c7e-7d54-47d2-9ca4-3d574f61f941 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:21.828147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb2c7e-7d54-47d2-9ca4-3d574f61f941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:21.829803 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb2c7e-7d54-47d2-9ca4-3d574f61f941 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:21.845941 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 972/1973] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:21.847201 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65110fd1-a5e2-4813-881e-faf6f3c1d95d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:21.859363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65110fd1-a5e2-4813-881e-faf6f3c1d95d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:21.865114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba961a-5580-40cb-8e5d-8a4591d9e098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:21.865450 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba961a-5580-40cb-8e5d-8a4591d9e098 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:21.865744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba961a-5580-40cb-8e5d-8a4591d9e098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:21.866023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba961a-5580-40cb-8e5d-8a4591d9e098 admin admin] PATH_INFO: `/v3/projects/117523bc87394e1e80842aac0ace8280/users/018af68e9f8543adbc15b66adb788293/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:21.868523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba961a-5580-40cb-8e5d-8a4591d9e098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=117523bc87394e1e80842aac0ace8280, user_id=018af68e9f8543adbc15b66adb788293, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:21.871990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba961a-5580-40cb-8e5d-8a4591d9e098 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:21.884423 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1002/1974] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:21 2022] PUT /identity/v3/projects/117523bc87394e1e80842aac0ace8280/users/018af68e9f8543adbc15b66adb788293/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:21.895668 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02950e92-c92c-41d9-b39a-a02af947334d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:21.895974 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02950e92-c92c-41d9-b39a-a02af947334d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:21.895974 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02950e92-c92c-41d9-b39a-a02af947334d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:21.928896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-02950e92-c92c-41d9-b39a-a02af947334d None None] MFA Rules not processed for user `018af68e9f8543adbc15b66adb788293`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:21.930880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-02950e92-c92c-41d9-b39a-a02af947334d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:22.024828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 973/1975] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:22.038705 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba961a-5580-40cb-8e5d-8a4591d9e098 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:22.042240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9bba961a-5580-40cb-8e5d-8a4591d9e098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:22.056848 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:22.106174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba961a-5580-40cb-8e5d-8a4591d9e098 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:22.113170 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629c84a-2c44-49eb-8f95-92e4eb4f3614 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:22.113842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629c84a-2c44-49eb-8f95-92e4eb4f3614 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:22.114128 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629c84a-2c44-49eb-8f95-92e4eb4f3614 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:22.114408 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629c84a-2c44-49eb-8f95-92e4eb4f3614 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:22.126490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629c84a-2c44-49eb-8f95-92e4eb4f3614 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:22.127996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629c84a-2c44-49eb-8f95-92e4eb4f3614 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:22.140012 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1003/1976] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:22.281193 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02950e92-c92c-41d9-b39a-a02af947334d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:22.283446 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-02950e92-c92c-41d9-b39a-a02af947334d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:22.299304 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02950e92-c92c-41d9-b39a-a02af947334d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:22.303605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054aa35-4b2b-445e-b627-a8b9a11b1d6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:22.304283 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054aa35-4b2b-445e-b627-a8b9a11b1d6a service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:22.304575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054aa35-4b2b-445e-b627-a8b9a11b1d6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:22.304860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054aa35-4b2b-445e-b627-a8b9a11b1d6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:22.306753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2054aa35-4b2b-445e-b627-a8b9a11b1d6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:22.343491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629c84a-2c44-49eb-8f95-92e4eb4f3614 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:22.345103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b629c84a-2c44-49eb-8f95-92e4eb4f3614 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:22.348599 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:22.389106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054aa35-4b2b-445e-b627-a8b9a11b1d6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:22.391916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054aa35-4b2b-445e-b627-a8b9a11b1d6a service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:22.407193 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 974/1977] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:09:22.450512 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629c84a-2c44-49eb-8f95-92e4eb4f3614 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:22.457895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32ede9-0a80-4d63-9ff8-89df0df5f7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:22.458908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32ede9-0a80-4d63-9ff8-89df0df5f7bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:22.459392 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32ede9-0a80-4d63-9ff8-89df0df5f7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:22.459751 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32ede9-0a80-4d63-9ff8-89df0df5f7bd admin admin] PATH_INFO: `/v3/projects/443b626cd6b34a23bfeff46ca00706ad` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:22.461137 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32ede9-0a80-4d63-9ff8-89df0df5f7bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=443b626cd6b34a23bfeff46ca00706ad)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:22.463180 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32ede9-0a80-4d63-9ff8-89df0df5f7bd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:22.629387 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9a32ede9-0a80-4d63-9ff8-89df0df5f7bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '443b626cd6b34a23bfeff46ca00706ad', 'request_id': 'req-9a32ede9-0a80-4d63-9ff8-89df0df5f7bd'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:22.699228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9a32ede9-0a80-4d63-9ff8-89df0df5f7bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '443b626cd6b34a23bfeff46ca00706ad', 'request_id': 'req-9a32ede9-0a80-4d63-9ff8-89df0df5f7bd'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:22.709949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9a32ede9-0a80-4d63-9ff8-89df0df5f7bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '443b626cd6b34a23bfeff46ca00706ad', 'request_id': 'req-9a32ede9-0a80-4d63-9ff8-89df0df5f7bd'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:22.721636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9a32ede9-0a80-4d63-9ff8-89df0df5f7bd admin admin] The token cache is being invalidate because project 443b626cd6b34a23bfeff46ca00706ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:22.721923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9a32ede9-0a80-4d63-9ff8-89df0df5f7bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:22.722587 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9a32ede9-0a80-4d63-9ff8-89df0df5f7bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:22.724219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1004/1978] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:22 2022] DELETE /identity/v3/projects/443b626cd6b34a23bfeff46ca00706ad => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:22.728159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054aa35-4b2b-445e-b627-a8b9a11b1d6a service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:22.740161 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054aa35-4b2b-445e-b627-a8b9a11b1d6a service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:22.803244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28479648-de09-408a-9f1a-78d52e6c07ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:22.804133 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28479648-de09-408a-9f1a-78d52e6c07ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:22.804422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28479648-de09-408a-9f1a-78d52e6c07ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:22.804704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28479648-de09-408a-9f1a-78d52e6c07ab admin admin] PATH_INFO: `/v3/users/67742c4e9d464f5cb76f2396c4316305` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:22.805975 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28479648-de09-408a-9f1a-78d52e6c07ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=67742c4e9d464f5cb76f2396c4316305)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:22.807963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28479648-de09-408a-9f1a-78d52e6c07ab admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:23.005322 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-28479648-de09-408a-9f1a-78d52e6c07ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67742c4e9d464f5cb76f2396c4316305'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:23.014902 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-28479648-de09-408a-9f1a-78d52e6c07ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67742c4e9d464f5cb76f2396c4316305'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:23.015855 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-28479648-de09-408a-9f1a-78d52e6c07ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67742c4e9d464f5cb76f2396c4316305'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:23.016596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-28479648-de09-408a-9f1a-78d52e6c07ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67742c4e9d464f5cb76f2396c4316305'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:23.032437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-28479648-de09-408a-9f1a-78d52e6c07ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67742c4e9d464f5cb76f2396c4316305'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:23.118298 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 975/1979] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:22 2022] DELETE /identity/v3/users/67742c4e9d464f5cb76f2396c4316305 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:23.673325 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32ede9-0a80-4d63-9ff8-89df0df5f7bd admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:23.674811 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9a32ede9-0a80-4d63-9ff8-89df0df5f7bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:23.706302 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32ede9-0a80-4d63-9ff8-89df0df5f7bd admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:23.775273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:23.807911 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c31c5f-320e-45b6-bfe8-5b6badea7dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:23.809382 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c31c5f-320e-45b6-bfe8-5b6badea7dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:23.809659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c31c5f-320e-45b6-bfe8-5b6badea7dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:23.809937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c31c5f-320e-45b6-bfe8-5b6badea7dc2 admin admin] PATH_INFO: `/v3/projects/b6f25f7861584475903616c93bdced84` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:23.811347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c31c5f-320e-45b6-bfe8-5b6badea7dc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6f25f7861584475903616c93bdced84)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:23.813374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c31c5f-320e-45b6-bfe8-5b6badea7dc2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:23.928405 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-20c31c5f-320e-45b6-bfe8-5b6badea7dc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6f25f7861584475903616c93bdced84', 'request_id': 'req-20c31c5f-320e-45b6-bfe8-5b6badea7dc2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:24.002373 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-20c31c5f-320e-45b6-bfe8-5b6badea7dc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6f25f7861584475903616c93bdced84', 'request_id': 'req-20c31c5f-320e-45b6-bfe8-5b6badea7dc2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:24.009998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-20c31c5f-320e-45b6-bfe8-5b6badea7dc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6f25f7861584475903616c93bdced84', 'request_id': 'req-20c31c5f-320e-45b6-bfe8-5b6badea7dc2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:24.021361 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-20c31c5f-320e-45b6-bfe8-5b6badea7dc2 admin admin] The token cache is being invalidate because project b6f25f7861584475903616c93bdced84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:24.021644 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-20c31c5f-320e-45b6-bfe8-5b6badea7dc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:24.022554 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-20c31c5f-320e-45b6-bfe8-5b6badea7dc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:24.024140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1005/1980] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:23 2022] DELETE /identity/v3/projects/b6f25f7861584475903616c93bdced84 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:24.029492 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049d3f0-441c-4b2a-badd-f2f002ab8b91 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:24.030107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049d3f0-441c-4b2a-badd-f2f002ab8b91 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:24.030107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049d3f0-441c-4b2a-badd-f2f002ab8b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:24.048860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c049d3f0-441c-4b2a-badd-f2f002ab8b91 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:24.050098 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c049d3f0-441c-4b2a-badd-f2f002ab8b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:24.108381 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:24.126892 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 976/1981] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:24.141312 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a639cb8-29f2-4bb2-8d96-46f98a6205d2 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:24.141609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a639cb8-29f2-4bb2-8d96-46f98a6205d2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:24.141895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a639cb8-29f2-4bb2-8d96-46f98a6205d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:24.162660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7a639cb8-29f2-4bb2-8d96-46f98a6205d2 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:24.164255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7a639cb8-29f2-4bb2-8d96-46f98a6205d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:24.173779 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1006/1982] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:24.178289 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049d3f0-441c-4b2a-badd-f2f002ab8b91 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:24.192983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049d3f0-441c-4b2a-badd-f2f002ab8b91 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:24.198280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651f37b-a208-4a5b-99e0-8d1c0cd7f441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:24.199220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651f37b-a208-4a5b-99e0-8d1c0cd7f441 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:24.199845 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651f37b-a208-4a5b-99e0-8d1c0cd7f441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:24.200363 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651f37b-a208-4a5b-99e0-8d1c0cd7f441 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:24.201587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651f37b-a208-4a5b-99e0-8d1c0cd7f441 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:09:24.202096 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651f37b-a208-4a5b-99e0-8d1c0cd7f441 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:24.204337 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651f37b-a208-4a5b-99e0-8d1c0cd7f441 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:24.221103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 977/1983] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:09:24.225245 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a639cb8-29f2-4bb2-8d96-46f98a6205d2 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:24.237543 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a639cb8-29f2-4bb2-8d96-46f98a6205d2 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:24.243284 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c0844-b230-47df-9195-6532c03a166a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:24.244187 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c0844-b230-47df-9195-6532c03a166a admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:24.244554 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c0844-b230-47df-9195-6532c03a166a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:24.244914 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c0844-b230-47df-9195-6532c03a166a admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:24.246039 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c0844-b230-47df-9195-6532c03a166a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:24.248660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c0844-b230-47df-9195-6532c03a166a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:24.386397 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1007/1984] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:24 2022] POST /identity/v3/projects => generated 383 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:24.391721 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651f37b-a208-4a5b-99e0-8d1c0cd7f441 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:24.402784 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651f37b-a208-4a5b-99e0-8d1c0cd7f441 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:24.460960 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59448c0e-3110-464d-9aae-ce3c069c55cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:24.461570 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59448c0e-3110-464d-9aae-ce3c069c55cb admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:24.461897 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59448c0e-3110-464d-9aae-ce3c069c55cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:24.462183 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59448c0e-3110-464d-9aae-ce3c069c55cb admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:24.462899 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59448c0e-3110-464d-9aae-ce3c069c55cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:24.464881 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59448c0e-3110-464d-9aae-ce3c069c55cb admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:24.585476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 978/1985] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:24 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:24.594285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c0844-b230-47df-9195-6532c03a166a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:24.604714 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c0844-b230-47df-9195-6532c03a166a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:24.609692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48d883-9d61-4df3-b1ee-960610198843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:24.610414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48d883-9d61-4df3-b1ee-960610198843 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:24.610698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48d883-9d61-4df3-b1ee-960610198843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:24.610975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48d883-9d61-4df3-b1ee-960610198843 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:24.611832 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48d883-9d61-4df3-b1ee-960610198843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:24.613462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48d883-9d61-4df3-b1ee-960610198843 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:24.628496 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1008/1986] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:24.630633 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59448c0e-3110-464d-9aae-ce3c069c55cb admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:24.642020 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59448c0e-3110-464d-9aae-ce3c069c55cb admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:24.649491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505e3ac-cef0-4398-9ad2-081c82b24e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:24.650473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505e3ac-cef0-4398-9ad2-081c82b24e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:24.650473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505e3ac-cef0-4398-9ad2-081c82b24e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:24.653033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505e3ac-cef0-4398-9ad2-081c82b24e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:24.653033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505e3ac-cef0-4398-9ad2-081c82b24e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:24.654064 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505e3ac-cef0-4398-9ad2-081c82b24e8b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:24.665006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 979/1987] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:24.669903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48d883-9d61-4df3-b1ee-960610198843 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:24.680723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48d883-9d61-4df3-b1ee-960610198843 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:24.691950 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafd526-0be2-4a2f-b830-fa5ccb8d8d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:24.692855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafd526-0be2-4a2f-b830-fa5ccb8d8d3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:24.693145 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafd526-0be2-4a2f-b830-fa5ccb8d8d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:24.693555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafd526-0be2-4a2f-b830-fa5ccb8d8d3f admin admin] PATH_INFO: `/v3/projects/594fb83650d14cbb8b0bd1c63db78ec9/users/af3b601315e043d8a028a72b228caa9f/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:24.715675 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeafd526-0be2-4a2f-b830-fa5ccb8d8d3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=594fb83650d14cbb8b0bd1c63db78ec9, user_id=af3b601315e043d8a028a72b228caa9f, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:24.718022 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeafd526-0be2-4a2f-b830-fa5ccb8d8d3f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:24.796809 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1009/1988] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:24 2022] PUT /identity/v3/projects/594fb83650d14cbb8b0bd1c63db78ec9/users/af3b601315e043d8a028a72b228caa9f/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:24.809174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8a002-10ab-4e53-b1fc-2f8cd5b5097b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:24.809464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8a002-10ab-4e53-b1fc-2f8cd5b5097b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:24.809749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8a002-10ab-4e53-b1fc-2f8cd5b5097b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:24.846742 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-5fd8a002-10ab-4e53-b1fc-2f8cd5b5097b None None] MFA Rules not processed for user `af3b601315e043d8a028a72b228caa9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:24.848597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5fd8a002-10ab-4e53-b1fc-2f8cd5b5097b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:24.900373 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafd526-0be2-4a2f-b830-fa5ccb8d8d3f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:24.903405 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-eeafd526-0be2-4a2f-b830-fa5ccb8d8d3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:24.982966 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 980/1989] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:24.990000 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8a002-10ab-4e53-b1fc-2f8cd5b5097b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:24.999236 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafd526-0be2-4a2f-b830-fa5ccb8d8d3f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:25.015353 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb45afa-186c-4046-b976-8a9162595aa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:25.016401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb45afa-186c-4046-b976-8a9162595aa5 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:25.016911 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb45afa-186c-4046-b976-8a9162595aa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:25.017413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb45afa-186c-4046-b976-8a9162595aa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:25.020315 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7bb45afa-186c-4046-b976-8a9162595aa5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:25.072733 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:25.083346 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:25.109005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8a002-10ab-4e53-b1fc-2f8cd5b5097b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:25.117097 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ffc76-16f3-4b84-b362-ef79dc07e2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:25.118358 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ffc76-16f3-4b84-b362-ef79dc07e2df admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:25.119074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ffc76-16f3-4b84-b362-ef79dc07e2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:25.119738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ffc76-16f3-4b84-b362-ef79dc07e2df admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:25.122045 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ffc76-16f3-4b84-b362-ef79dc07e2df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:25.124819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ffc76-16f3-4b84-b362-ef79dc07e2df admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:25.134528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb45afa-186c-4046-b976-8a9162595aa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:25.136203 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb45afa-186c-4046-b976-8a9162595aa5 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:25.156626 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1010/1990] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:24 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:09:25.256032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 981/1991] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:24 2022] POST /identity/v3/projects => generated 383 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:25.260509 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb45afa-186c-4046-b976-8a9162595aa5 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:25.271946 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb45afa-186c-4046-b976-8a9162595aa5 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:25.361776 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37deeeea-a928-47ac-9028-0471eed61c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:25.362472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37deeeea-a928-47ac-9028-0471eed61c07 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:25.362757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37deeeea-a928-47ac-9028-0471eed61c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:25.365484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37deeeea-a928-47ac-9028-0471eed61c07 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:25.365484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37deeeea-a928-47ac-9028-0471eed61c07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:25.365969 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37deeeea-a928-47ac-9028-0471eed61c07 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:25.490316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1011/1992] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:25 2022] POST /identity/v3/users => generated 298 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:09:25.494234 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ffc76-16f3-4b84-b362-ef79dc07e2df admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:25.505343 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ffc76-16f3-4b84-b362-ef79dc07e2df admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:25.510362 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197824ed-d29e-40fe-b976-493852c0aa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:25.511148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197824ed-d29e-40fe-b976-493852c0aa14 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:25.511483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197824ed-d29e-40fe-b976-493852c0aa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:25.512282 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197824ed-d29e-40fe-b976-493852c0aa14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:25.513024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197824ed-d29e-40fe-b976-493852c0aa14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:25.514745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197824ed-d29e-40fe-b976-493852c0aa14 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:25.526534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 982/1993] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:25.535740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37deeeea-a928-47ac-9028-0471eed61c07 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:25.549753 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37deeeea-a928-47ac-9028-0471eed61c07 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:25.556156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfda44c-d716-4c55-bdde-b4a9e230304f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:25.557092 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfda44c-d716-4c55-bdde-b4a9e230304f admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:25.557374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfda44c-d716-4c55-bdde-b4a9e230304f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:25.557647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfda44c-d716-4c55-bdde-b4a9e230304f admin admin] PATH_INFO: `/v3/projects/cf5cb479294045d0a9dfa2f39a3942eb/users/2e4d6cde002c48f581d5d6d9677d24d4/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:25.577902 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfda44c-d716-4c55-bdde-b4a9e230304f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf5cb479294045d0a9dfa2f39a3942eb, user_id=2e4d6cde002c48f581d5d6d9677d24d4, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:25.580720 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfda44c-d716-4c55-bdde-b4a9e230304f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:25.671505 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1012/1994] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:25 2022] PUT /identity/v3/projects/cf5cb479294045d0a9dfa2f39a3942eb/users/2e4d6cde002c48f581d5d6d9677d24d4/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:25.675857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197824ed-d29e-40fe-b976-493852c0aa14 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:25.686777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197824ed-d29e-40fe-b976-493852c0aa14 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:25.774297 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7766c0-4c5a-4402-a518-c609df5fd06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:25.775460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7766c0-4c5a-4402-a518-c609df5fd06b admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:25.775921 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7766c0-4c5a-4402-a518-c609df5fd06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:25.776361 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7766c0-4c5a-4402-a518-c609df5fd06b admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:25.778359 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7766c0-4c5a-4402-a518-c609df5fd06b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:25.780730 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7766c0-4c5a-4402-a518-c609df5fd06b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:25.799075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 983/1995] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:25.804531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfda44c-d716-4c55-bdde-b4a9e230304f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:25.815252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfda44c-d716-4c55-bdde-b4a9e230304f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:25.820562 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf37b8e-6140-4c9e-8ed8-725ed8b4be80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:25.821479 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf37b8e-6140-4c9e-8ed8-725ed8b4be80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:25.821771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf37b8e-6140-4c9e-8ed8-725ed8b4be80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:25.822057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf37b8e-6140-4c9e-8ed8-725ed8b4be80 admin admin] PATH_INFO: `/v3/projects/cf5cb479294045d0a9dfa2f39a3942eb/users/2e4d6cde002c48f581d5d6d9677d24d4/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:25.824370 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf37b8e-6140-4c9e-8ed8-725ed8b4be80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf5cb479294045d0a9dfa2f39a3942eb, user_id=2e4d6cde002c48f581d5d6d9677d24d4, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:25.828275 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf37b8e-6140-4c9e-8ed8-725ed8b4be80 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:25.851006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1013/1996] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:25 2022] PUT /identity/v3/projects/cf5cb479294045d0a9dfa2f39a3942eb/users/2e4d6cde002c48f581d5d6d9677d24d4/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:25.865410 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6bb18-4020-432b-be31-0e1843839978 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:25.865944 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6bb18-4020-432b-be31-0e1843839978 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:25.866355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6bb18-4020-432b-be31-0e1843839978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:25.903764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-52f6bb18-4020-432b-be31-0e1843839978 None None] MFA Rules not processed for user `2e4d6cde002c48f581d5d6d9677d24d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:25.905141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-52f6bb18-4020-432b-be31-0e1843839978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:26.018118 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 984/1997] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:09:26.089052 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf37b8e-6140-4c9e-8ed8-725ed8b4be80 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:26.166972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf37b8e-6140-4c9e-8ed8-725ed8b4be80 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:26.171355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb671266-faa5-45b3-ac37-754f2ef5491e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:26.172019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb671266-faa5-45b3-ac37-754f2ef5491e service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:26.172306 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb671266-faa5-45b3-ac37-754f2ef5491e service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:26.172588 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb671266-faa5-45b3-ac37-754f2ef5491e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:26.182763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb671266-faa5-45b3-ac37-754f2ef5491e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:26.185027 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb671266-faa5-45b3-ac37-754f2ef5491e service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:26.195853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1014/1998] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:09:26.443004 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6bb18-4020-432b-be31-0e1843839978 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:26.461146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6bb18-4020-432b-be31-0e1843839978 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:26.467412 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f08e01-34b8-4c1a-ade3-36015841204e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:26.468841 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f08e01-34b8-4c1a-ade3-36015841204e service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:26.469331 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f08e01-34b8-4c1a-ade3-36015841204e service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:26.470186 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f08e01-34b8-4c1a-ade3-36015841204e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:26.541874 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:26.567192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f08e01-34b8-4c1a-ade3-36015841204e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:26.569083 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f08e01-34b8-4c1a-ade3-36015841204e service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:26.581793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 985/1999] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:27.318505 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb671266-faa5-45b3-ac37-754f2ef5491e service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:27.339214 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:27.358425 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f08e01-34b8-4c1a-ade3-36015841204e service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:27.360534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-32f08e01-34b8-4c1a-ade3-36015841204e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:27.408762 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb671266-faa5-45b3-ac37-754f2ef5491e service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:27.427293 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d2808-ae98-4a1a-9ead-036d0a7b5840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:27.428670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d2808-ae98-4a1a-9ead-036d0a7b5840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:27.429537 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d2808-ae98-4a1a-9ead-036d0a7b5840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:27.429812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d2808-ae98-4a1a-9ead-036d0a7b5840 admin admin] PATH_INFO: `/v3/users/af3b601315e043d8a028a72b228caa9f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:27.431430 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d2808-ae98-4a1a-9ead-036d0a7b5840 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af3b601315e043d8a028a72b228caa9f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:27.439433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d2808-ae98-4a1a-9ead-036d0a7b5840 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:27.523800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f08e01-34b8-4c1a-ade3-36015841204e service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:27.530866 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44a333-015c-47f4-aa5d-5500711f305b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:27.532788 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44a333-015c-47f4-aa5d-5500711f305b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:27.534067 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44a333-015c-47f4-aa5d-5500711f305b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:27.534675 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44a333-015c-47f4-aa5d-5500711f305b admin admin] PATH_INFO: `/v3/users/3fbf4ed834704615b1ba01111f880b91` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:27.543846 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44a333-015c-47f4-aa5d-5500711f305b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fbf4ed834704615b1ba01111f880b91)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:27.549036 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44a333-015c-47f4-aa5d-5500711f305b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:27.813634 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0f44a333-015c-47f4-aa5d-5500711f305b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fbf4ed834704615b1ba01111f880b91'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:27.823440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0f44a333-015c-47f4-aa5d-5500711f305b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fbf4ed834704615b1ba01111f880b91'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:27.824308 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0f44a333-015c-47f4-aa5d-5500711f305b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fbf4ed834704615b1ba01111f880b91'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:27.824995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0f44a333-015c-47f4-aa5d-5500711f305b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fbf4ed834704615b1ba01111f880b91'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:27.833766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9d7d2808-ae98-4a1a-9ead-036d0a7b5840 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af3b601315e043d8a028a72b228caa9f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:27.834604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9d7d2808-ae98-4a1a-9ead-036d0a7b5840 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af3b601315e043d8a028a72b228caa9f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:27.841682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0f44a333-015c-47f4-aa5d-5500711f305b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fbf4ed834704615b1ba01111f880b91'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:27.852144 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9d7d2808-ae98-4a1a-9ead-036d0a7b5840 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af3b601315e043d8a028a72b228caa9f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:27.852914 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9d7d2808-ae98-4a1a-9ead-036d0a7b5840 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af3b601315e043d8a028a72b228caa9f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:27.861420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9d7d2808-ae98-4a1a-9ead-036d0a7b5840 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af3b601315e043d8a028a72b228caa9f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:27.930485 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 986/2000] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:27 2022] DELETE /identity/v3/users/3fbf4ed834704615b1ba01111f880b91 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:27.936290 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8849f9f-3f47-4b70-a159-5b28dabc8055 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:27.938046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8849f9f-3f47-4b70-a159-5b28dabc8055 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:27.938046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8849f9f-3f47-4b70-a159-5b28dabc8055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:27.968034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f8849f9f-3f47-4b70-a159-5b28dabc8055 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:27.975086 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f8849f9f-3f47-4b70-a159-5b28dabc8055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:28.010586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1015/2001] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:27 2022] DELETE /identity/v3/users/af3b601315e043d8a028a72b228caa9f => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:28.018912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3f9f1-997c-4c63-ac1e-39a0d91a9274 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:28.018912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3f9f1-997c-4c63-ac1e-39a0d91a9274 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:28.018912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3f9f1-997c-4c63-ac1e-39a0d91a9274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:28.056283 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-1fc3f9f1-997c-4c63-ac1e-39a0d91a9274 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:28.057565 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1fc3f9f1-997c-4c63-ac1e-39a0d91a9274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:28.087809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:28.108772 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 987/2002] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:09:28.155081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8849f9f-3f47-4b70-a159-5b28dabc8055 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:28.157284 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f8849f9f-3f47-4b70-a159-5b28dabc8055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:28.172704 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1016/2003] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:28.190090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3f9f1-997c-4c63-ac1e-39a0d91a9274 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:28.192158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1fc3f9f1-997c-4c63-ac1e-39a0d91a9274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:28.295739 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:28.307033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8849f9f-3f47-4b70-a159-5b28dabc8055 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:28.336966 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3f9f1-997c-4c63-ac1e-39a0d91a9274 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:28.347452 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f361e-b233-4c94-9afd-f17e00c4a355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:28.348620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f361e-b233-4c94-9afd-f17e00c4a355 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:28.349075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f361e-b233-4c94-9afd-f17e00c4a355 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:28.349703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f361e-b233-4c94-9afd-f17e00c4a355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:28.371488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f361e-b233-4c94-9afd-f17e00c4a355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:28.373176 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f361e-b233-4c94-9afd-f17e00c4a355 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:28.403342 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1017/2004] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:09:28.440146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313cd59-d456-4448-83a9-64f8d86bb3e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:28.440847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313cd59-d456-4448-83a9-64f8d86bb3e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:28.441141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313cd59-d456-4448-83a9-64f8d86bb3e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:28.441426 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313cd59-d456-4448-83a9-64f8d86bb3e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:28.469508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313cd59-d456-4448-83a9-64f8d86bb3e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:28.471123 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313cd59-d456-4448-83a9-64f8d86bb3e8 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:28.482819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 988/2005] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:29.130960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f361e-b233-4c94-9afd-f17e00c4a355 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:29.133264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-537f361e-b233-4c94-9afd-f17e00c4a355 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:29.160748 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f361e-b233-4c94-9afd-f17e00c4a355 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:29.174003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd195d7-e5ef-4a32-8fe0-a92ee0cd456b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:29.174003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd195d7-e5ef-4a32-8fe0-a92ee0cd456b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:29.174003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd195d7-e5ef-4a32-8fe0-a92ee0cd456b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:29.174003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd195d7-e5ef-4a32-8fe0-a92ee0cd456b admin admin] PATH_INFO: `/v3/projects/594fb83650d14cbb8b0bd1c63db78ec9` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:29.174679 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd195d7-e5ef-4a32-8fe0-a92ee0cd456b admin admin] RBAC: Authorizing `identity:delete_project(project_id=594fb83650d14cbb8b0bd1c63db78ec9)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:29.176601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd195d7-e5ef-4a32-8fe0-a92ee0cd456b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:29.337016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1dd195d7-e5ef-4a32-8fe0-a92ee0cd456b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '594fb83650d14cbb8b0bd1c63db78ec9', 'request_id': 'req-1dd195d7-e5ef-4a32-8fe0-a92ee0cd456b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:29.398192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1dd195d7-e5ef-4a32-8fe0-a92ee0cd456b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '594fb83650d14cbb8b0bd1c63db78ec9', 'request_id': 'req-1dd195d7-e5ef-4a32-8fe0-a92ee0cd456b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:29.410708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1dd195d7-e5ef-4a32-8fe0-a92ee0cd456b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '594fb83650d14cbb8b0bd1c63db78ec9', 'request_id': 'req-1dd195d7-e5ef-4a32-8fe0-a92ee0cd456b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:29.443229 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1dd195d7-e5ef-4a32-8fe0-a92ee0cd456b admin admin] The token cache is being invalidate because project 594fb83650d14cbb8b0bd1c63db78ec9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:29.443711 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1dd195d7-e5ef-4a32-8fe0-a92ee0cd456b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:29.444406 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1dd195d7-e5ef-4a32-8fe0-a92ee0cd456b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:29.450534 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1018/2006] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:29 2022] DELETE /identity/v3/projects/594fb83650d14cbb8b0bd1c63db78ec9 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:29.456812 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313cd59-d456-4448-83a9-64f8d86bb3e8 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:29.472734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313cd59-d456-4448-83a9-64f8d86bb3e8 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:29.583201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f305768-6f6f-45c3-8ee5-e7a3cefd12e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:29.584475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f305768-6f6f-45c3-8ee5-e7a3cefd12e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:29.588736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f305768-6f6f-45c3-8ee5-e7a3cefd12e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:29.588736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f305768-6f6f-45c3-8ee5-e7a3cefd12e8 admin admin] PATH_INFO: `/v3/users/2e4d6cde002c48f581d5d6d9677d24d4` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:29.588736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f305768-6f6f-45c3-8ee5-e7a3cefd12e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e4d6cde002c48f581d5d6d9677d24d4)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:29.590612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f305768-6f6f-45c3-8ee5-e7a3cefd12e8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:29.911364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3f305768-6f6f-45c3-8ee5-e7a3cefd12e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e4d6cde002c48f581d5d6d9677d24d4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:29.935439 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3f305768-6f6f-45c3-8ee5-e7a3cefd12e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e4d6cde002c48f581d5d6d9677d24d4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:29.935439 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3f305768-6f6f-45c3-8ee5-e7a3cefd12e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e4d6cde002c48f581d5d6d9677d24d4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:29.936140 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3f305768-6f6f-45c3-8ee5-e7a3cefd12e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e4d6cde002c48f581d5d6d9677d24d4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:29.942235 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:29.962783 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3f305768-6f6f-45c3-8ee5-e7a3cefd12e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e4d6cde002c48f581d5d6d9677d24d4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:30.029710 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd195d7-e5ef-4a32-8fe0-a92ee0cd456b admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:30.029710 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1dd195d7-e5ef-4a32-8fe0-a92ee0cd456b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:30.044811 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd195d7-e5ef-4a32-8fe0-a92ee0cd456b admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:30.055890 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 989/2007] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:29 2022] DELETE /identity/v3/users/2e4d6cde002c48f581d5d6d9677d24d4 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:30.070214 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c218fe-ee93-46b7-aef8-e6b7673e630e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:30.071360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c218fe-ee93-46b7-aef8-e6b7673e630e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:30.071924 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c218fe-ee93-46b7-aef8-e6b7673e630e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:30.072495 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c218fe-ee93-46b7-aef8-e6b7673e630e admin admin] PATH_INFO: `/v3/projects/04e5d96927b440c1b26aeb871cc8e86b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:30.076273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c218fe-ee93-46b7-aef8-e6b7673e630e admin admin] RBAC: Authorizing `identity:delete_project(project_id=04e5d96927b440c1b26aeb871cc8e86b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:30.081716 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c218fe-ee93-46b7-aef8-e6b7673e630e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:30.293607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b0c218fe-ee93-46b7-aef8-e6b7673e630e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04e5d96927b440c1b26aeb871cc8e86b', 'request_id': 'req-b0c218fe-ee93-46b7-aef8-e6b7673e630e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:30.398762 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b0c218fe-ee93-46b7-aef8-e6b7673e630e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04e5d96927b440c1b26aeb871cc8e86b', 'request_id': 'req-b0c218fe-ee93-46b7-aef8-e6b7673e630e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:30.409727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b0c218fe-ee93-46b7-aef8-e6b7673e630e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04e5d96927b440c1b26aeb871cc8e86b', 'request_id': 'req-b0c218fe-ee93-46b7-aef8-e6b7673e630e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:30.441975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b0c218fe-ee93-46b7-aef8-e6b7673e630e admin admin] The token cache is being invalidate because project 04e5d96927b440c1b26aeb871cc8e86b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:30.442617 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b0c218fe-ee93-46b7-aef8-e6b7673e630e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:30.443975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b0c218fe-ee93-46b7-aef8-e6b7673e630e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:30.446839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1019/2008] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:30 2022] DELETE /identity/v3/projects/04e5d96927b440c1b26aeb871cc8e86b => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:30.451099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f305768-6f6f-45c3-8ee5-e7a3cefd12e8 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:30.452894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3f305768-6f6f-45c3-8ee5-e7a3cefd12e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:30.592590 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f305768-6f6f-45c3-8ee5-e7a3cefd12e8 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:30.678164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12df24-f41f-420e-8632-c30578f49d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:30.679533 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12df24-f41f-420e-8632-c30578f49d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:30.680154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12df24-f41f-420e-8632-c30578f49d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:30.680642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12df24-f41f-420e-8632-c30578f49d9e admin admin] PATH_INFO: `/v3/users/018af68e9f8543adbc15b66adb788293` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:30.682204 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12df24-f41f-420e-8632-c30578f49d9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=018af68e9f8543adbc15b66adb788293)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:30.685026 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12df24-f41f-420e-8632-c30578f49d9e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:30.695315 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c218fe-ee93-46b7-aef8-e6b7673e630e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:30.700531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b0c218fe-ee93-46b7-aef8-e6b7673e630e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:30.729579 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c218fe-ee93-46b7-aef8-e6b7673e630e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:30.738690 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70ab60-0a63-4c4f-b5da-6c1df6279b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:30.740155 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70ab60-0a63-4c4f-b5da-6c1df6279b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:30.740604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70ab60-0a63-4c4f-b5da-6c1df6279b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:30.740898 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70ab60-0a63-4c4f-b5da-6c1df6279b8e admin admin] PATH_INFO: `/v3/projects/cf5cb479294045d0a9dfa2f39a3942eb` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:30.743040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd70ab60-0a63-4c4f-b5da-6c1df6279b8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf5cb479294045d0a9dfa2f39a3942eb)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:30.746330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd70ab60-0a63-4c4f-b5da-6c1df6279b8e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:31.029610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:31.047520 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-eb12df24-f41f-420e-8632-c30578f49d9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '018af68e9f8543adbc15b66adb788293'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:31.049093 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fd70ab60-0a63-4c4f-b5da-6c1df6279b8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf5cb479294045d0a9dfa2f39a3942eb', 'request_id': 'req-fd70ab60-0a63-4c4f-b5da-6c1df6279b8e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:31.061109 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-eb12df24-f41f-420e-8632-c30578f49d9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '018af68e9f8543adbc15b66adb788293'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:31.064084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-eb12df24-f41f-420e-8632-c30578f49d9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '018af68e9f8543adbc15b66adb788293'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:31.064084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-eb12df24-f41f-420e-8632-c30578f49d9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '018af68e9f8543adbc15b66adb788293'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:31.080580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-eb12df24-f41f-420e-8632-c30578f49d9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '018af68e9f8543adbc15b66adb788293'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:31.127224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fd70ab60-0a63-4c4f-b5da-6c1df6279b8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf5cb479294045d0a9dfa2f39a3942eb', 'request_id': 'req-fd70ab60-0a63-4c4f-b5da-6c1df6279b8e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:31.139140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fd70ab60-0a63-4c4f-b5da-6c1df6279b8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf5cb479294045d0a9dfa2f39a3942eb', 'request_id': 'req-fd70ab60-0a63-4c4f-b5da-6c1df6279b8e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:31.162247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fd70ab60-0a63-4c4f-b5da-6c1df6279b8e admin admin] The token cache is being invalidate because project cf5cb479294045d0a9dfa2f39a3942eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:31.162661 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fd70ab60-0a63-4c4f-b5da-6c1df6279b8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:31.164159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fd70ab60-0a63-4c4f-b5da-6c1df6279b8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:31.166247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1020/2009] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:30 2022] DELETE /identity/v3/projects/cf5cb479294045d0a9dfa2f39a3942eb => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:31.172941 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060b8dd-2864-44d0-8831-a68988f64a69 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:31.173659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060b8dd-2864-44d0-8831-a68988f64a69 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:31.174678 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060b8dd-2864-44d0-8831-a68988f64a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:31.197382 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2060b8dd-2864-44d0-8831-a68988f64a69 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:31.199608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2060b8dd-2864-44d0-8831-a68988f64a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:31.236960 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 990/2010] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:30 2022] DELETE /identity/v3/users/018af68e9f8543adbc15b66adb788293 => generated 0 bytes in 786 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:31.292300 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1021/2011] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:31.301425 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef674f8-6d6a-4561-adf4-a15e3028245c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:31.301870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef674f8-6d6a-4561-adf4-a15e3028245c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:31.302177 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef674f8-6d6a-4561-adf4-a15e3028245c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:31.309882 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:31.329273 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-cef674f8-6d6a-4561-adf4-a15e3028245c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:31.332470 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-cef674f8-6d6a-4561-adf4-a15e3028245c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:31.467360 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 991/2012] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:31.469685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060b8dd-2864-44d0-8831-a68988f64a69 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:31.499365 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060b8dd-2864-44d0-8831-a68988f64a69 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:31.499365 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c90a9-6eb8-4b3a-91a0-0d82f1b0c9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:31.499365 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c90a9-6eb8-4b3a-91a0-0d82f1b0c9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:31.499365 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c90a9-6eb8-4b3a-91a0-0d82f1b0c9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:31.499891 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c90a9-6eb8-4b3a-91a0-0d82f1b0c9a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:31.499891 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c90a9-6eb8-4b3a-91a0-0d82f1b0c9a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:09:31.499891 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c90a9-6eb8-4b3a-91a0-0d82f1b0c9a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:31.499891 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c90a9-6eb8-4b3a-91a0-0d82f1b0c9a8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:31.515491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1022/2013] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:09:31.516712 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef674f8-6d6a-4561-adf4-a15e3028245c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:31.526679 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef674f8-6d6a-4561-adf4-a15e3028245c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:31.537947 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247c173-80c0-4902-8f93-d17311f3777f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:31.538849 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247c173-80c0-4902-8f93-d17311f3777f admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:31.539240 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247c173-80c0-4902-8f93-d17311f3777f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:31.539747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247c173-80c0-4902-8f93-d17311f3777f admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:31.540542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247c173-80c0-4902-8f93-d17311f3777f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:31.542810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247c173-80c0-4902-8f93-d17311f3777f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:31.680334 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 992/2014] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:31 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:31.686285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c90a9-6eb8-4b3a-91a0-0d82f1b0c9a8 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:31.701196 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c90a9-6eb8-4b3a-91a0-0d82f1b0c9a8 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:31.782954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17b83c-48c7-4864-a8e2-68dea5e2932a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:31.783789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17b83c-48c7-4864-a8e2-68dea5e2932a admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:31.784079 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17b83c-48c7-4864-a8e2-68dea5e2932a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:31.784360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17b83c-48c7-4864-a8e2-68dea5e2932a admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:31.785162 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17b83c-48c7-4864-a8e2-68dea5e2932a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:31.787070 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17b83c-48c7-4864-a8e2-68dea5e2932a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:31.848779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247c173-80c0-4902-8f93-d17311f3777f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:31.854740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c247c173-80c0-4902-8f93-d17311f3777f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:31.876777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247c173-80c0-4902-8f93-d17311f3777f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:31.884286 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14a3bd-0b1c-4f73-8105-3e81295fed8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:31.885234 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14a3bd-0b1c-4f73-8105-3e81295fed8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:31.885470 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14a3bd-0b1c-4f73-8105-3e81295fed8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:31.885761 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14a3bd-0b1c-4f73-8105-3e81295fed8c admin admin] PATH_INFO: `/v3/projects/117523bc87394e1e80842aac0ace8280` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:31.887634 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14a3bd-0b1c-4f73-8105-3e81295fed8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=117523bc87394e1e80842aac0ace8280)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:31.900958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14a3bd-0b1c-4f73-8105-3e81295fed8c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:31.902637 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1023/2015] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:31 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:31.907199 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17b83c-48c7-4864-a8e2-68dea5e2932a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:31.921015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17b83c-48c7-4864-a8e2-68dea5e2932a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:31.927866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a9c08-f579-4d1a-b776-8162b457ed3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:31.928925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a9c08-f579-4d1a-b776-8162b457ed3e admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:31.929355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a9c08-f579-4d1a-b776-8162b457ed3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:31.930152 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a9c08-f579-4d1a-b776-8162b457ed3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:31.930741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a9c08-f579-4d1a-b776-8162b457ed3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:31.932582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a9c08-f579-4d1a-b776-8162b457ed3e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:31.946365 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1024/2016] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:31.951008 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a9c08-f579-4d1a-b776-8162b457ed3e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:31.961585 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a9c08-f579-4d1a-b776-8162b457ed3e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:31.969365 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5951e46-2aad-4207-ae49-bbeb7fa2408f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:31.970309 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5951e46-2aad-4207-ae49-bbeb7fa2408f admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:31.970827 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5951e46-2aad-4207-ae49-bbeb7fa2408f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:31.971122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5951e46-2aad-4207-ae49-bbeb7fa2408f admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:31.971939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5951e46-2aad-4207-ae49-bbeb7fa2408f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:31.973670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5951e46-2aad-4207-ae49-bbeb7fa2408f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:31.990165 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1025/2017] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:09:31.991143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5951e46-2aad-4207-ae49-bbeb7fa2408f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:32.023239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5951e46-2aad-4207-ae49-bbeb7fa2408f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:32.037471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-bf14a3bd-0b1c-4f73-8105-3e81295fed8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '117523bc87394e1e80842aac0ace8280', 'request_id': 'req-bf14a3bd-0b1c-4f73-8105-3e81295fed8c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:32.045782 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:32.096262 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb20c6-8e90-4e3b-94c6-3de606215514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:32.097304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb20c6-8e90-4e3b-94c6-3de606215514 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:32.098038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb20c6-8e90-4e3b-94c6-3de606215514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:32.098734 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb20c6-8e90-4e3b-94c6-3de606215514 admin admin] PATH_INFO: `/v3/projects/25afb3f130ba44fb9b363fda6f262d70/users/b6168a02789945d68a93c0c06d785753/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:32.099356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-bf14a3bd-0b1c-4f73-8105-3e81295fed8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '117523bc87394e1e80842aac0ace8280', 'request_id': 'req-bf14a3bd-0b1c-4f73-8105-3e81295fed8c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:32.109751 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-bf14a3bd-0b1c-4f73-8105-3e81295fed8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '117523bc87394e1e80842aac0ace8280', 'request_id': 'req-bf14a3bd-0b1c-4f73-8105-3e81295fed8c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:32.126822 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb20c6-8e90-4e3b-94c6-3de606215514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25afb3f130ba44fb9b363fda6f262d70, user_id=b6168a02789945d68a93c0c06d785753, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:32.129729 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb20c6-8e90-4e3b-94c6-3de606215514 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:32.131505 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-bf14a3bd-0b1c-4f73-8105-3e81295fed8c admin admin] The token cache is being invalidate because project 117523bc87394e1e80842aac0ace8280 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:32.132137 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-bf14a3bd-0b1c-4f73-8105-3e81295fed8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:32.134045 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-bf14a3bd-0b1c-4f73-8105-3e81295fed8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:32.141830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 993/2018] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:31 2022] DELETE /identity/v3/projects/117523bc87394e1e80842aac0ace8280 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:32.144635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce71cc-a458-4e1d-bebd-589fb4c2885c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:32.147042 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce71cc-a458-4e1d-bebd-589fb4c2885c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:32.147042 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce71cc-a458-4e1d-bebd-589fb4c2885c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:32.185443 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-86ce71cc-a458-4e1d-bebd-589fb4c2885c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:32.185443 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-86ce71cc-a458-4e1d-bebd-589fb4c2885c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:32.185443 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 994/2019] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:09:32.194537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a55da6-7a77-4e68-8b1a-cd3abf180578 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:32.195274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a55da6-7a77-4e68-8b1a-cd3abf180578 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:32.195939 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a55da6-7a77-4e68-8b1a-cd3abf180578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:32.219058 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-63a55da6-7a77-4e68-8b1a-cd3abf180578 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:32.220315 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-63a55da6-7a77-4e68-8b1a-cd3abf180578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:32.229767 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 995/2020] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:09:32.234060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a55da6-7a77-4e68-8b1a-cd3abf180578 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:32.242661 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1026/2021] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:31 2022] PUT /identity/v3/projects/25afb3f130ba44fb9b363fda6f262d70/users/b6168a02789945d68a93c0c06d785753/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:32.250340 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a55da6-7a77-4e68-8b1a-cd3abf180578 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:32.257148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be90c0b-97ef-42a6-a373-cd4656c379ed None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:32.258142 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be90c0b-97ef-42a6-a373-cd4656c379ed None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:32.258601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be90c0b-97ef-42a6-a373-cd4656c379ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:32.296656 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5be90c0b-97ef-42a6-a373-cd4656c379ed None None] MFA Rules not processed for user `b6168a02789945d68a93c0c06d785753`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:32.297867 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5be90c0b-97ef-42a6-a373-cd4656c379ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:32.311762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:32.323736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73596496-d32d-4be6-a1db-b8a3f749777b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:32.325210 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73596496-d32d-4be6-a1db-b8a3f749777b admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:32.325709 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73596496-d32d-4be6-a1db-b8a3f749777b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:32.326058 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73596496-d32d-4be6-a1db-b8a3f749777b admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:32.326808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73596496-d32d-4be6-a1db-b8a3f749777b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:09:32.327130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73596496-d32d-4be6-a1db-b8a3f749777b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:32.328796 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73596496-d32d-4be6-a1db-b8a3f749777b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:32.341377 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 996/2022] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 19:09:32.346210 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73596496-d32d-4be6-a1db-b8a3f749777b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:32.358943 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73596496-d32d-4be6-a1db-b8a3f749777b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:32.364401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21012fce-7082-41e0-a0e9-5cf33318ffe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:32.365079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21012fce-7082-41e0-a0e9-5cf33318ffe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:32.365371 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21012fce-7082-41e0-a0e9-5cf33318ffe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:32.365657 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21012fce-7082-41e0-a0e9-5cf33318ffe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:32.366363 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21012fce-7082-41e0-a0e9-5cf33318ffe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:32.368420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21012fce-7082-41e0-a0e9-5cf33318ffe2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:32.378029 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1027/2023] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:32.390268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be90c0b-97ef-42a6-a373-cd4656c379ed None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:32.391948 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5be90c0b-97ef-42a6-a373-cd4656c379ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:32.453057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be90c0b-97ef-42a6-a373-cd4656c379ed None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:32.457710 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55692b7c-f9b7-416a-8405-a39200b4bbca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:32.458360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55692b7c-f9b7-416a-8405-a39200b4bbca service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:32.458648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55692b7c-f9b7-416a-8405-a39200b4bbca service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:32.458969 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55692b7c-f9b7-416a-8405-a39200b4bbca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:32.469430 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55692b7c-f9b7-416a-8405-a39200b4bbca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:32.470880 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55692b7c-f9b7-416a-8405-a39200b4bbca service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:32.480787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1028/2024] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:32.484509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 997/2025] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:32 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:09:32.489037 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55692b7c-f9b7-416a-8405-a39200b4bbca service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:32.507568 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55692b7c-f9b7-416a-8405-a39200b4bbca service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:32.550202 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:32.579349 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb58ae-f865-4657-8870-fb48ed9218fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:32.580038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb58ae-f865-4657-8870-fb48ed9218fb admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:32.580332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb58ae-f865-4657-8870-fb48ed9218fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:32.580614 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb58ae-f865-4657-8870-fb48ed9218fb admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:32.581334 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabb58ae-f865-4657-8870-fb48ed9218fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:32.583389 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabb58ae-f865-4657-8870-fb48ed9218fb admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:32.703015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1029/2026] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:32 2022] POST /identity/v3/users => generated 288 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:32.708974 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21012fce-7082-41e0-a0e9-5cf33318ffe2 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:32.725240 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21012fce-7082-41e0-a0e9-5cf33318ffe2 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:32.730414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523946d4-ff94-45b1-a928-cdbd6694bdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:32.731111 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523946d4-ff94-45b1-a928-cdbd6694bdcc admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:32.731470 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523946d4-ff94-45b1-a928-cdbd6694bdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:32.731755 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523946d4-ff94-45b1-a928-cdbd6694bdcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:32.732492 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523946d4-ff94-45b1-a928-cdbd6694bdcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:32.734115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523946d4-ff94-45b1-a928-cdbd6694bdcc admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:32.745178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 998/2027] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:09:32.750642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb58ae-f865-4657-8870-fb48ed9218fb admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:32.765144 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb58ae-f865-4657-8870-fb48ed9218fb admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:32.774764 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4e1f7-1410-452c-aa0c-a9af5012d307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:32.775550 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4e1f7-1410-452c-aa0c-a9af5012d307 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:32.775903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4e1f7-1410-452c-aa0c-a9af5012d307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:32.776297 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4e1f7-1410-452c-aa0c-a9af5012d307 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:32.777571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4e1f7-1410-452c-aa0c-a9af5012d307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:32.779862 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4e1f7-1410-452c-aa0c-a9af5012d307 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:32.791233 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1030/2028] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:32.797342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523946d4-ff94-45b1-a928-cdbd6694bdcc admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:32.809779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523946d4-ff94-45b1-a928-cdbd6694bdcc admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:32.815544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a3ee5-0738-4551-b4a7-631f4f0339aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:32.816033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a3ee5-0738-4551-b4a7-631f4f0339aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:32.819091 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a3ee5-0738-4551-b4a7-631f4f0339aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:32.819091 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a3ee5-0738-4551-b4a7-631f4f0339aa admin admin] PATH_INFO: `/v3/projects/21d050a2697746719c75641f70da0cfd/users/f393f3fac09d40aa87290de0f46040b2/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:32.840910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a3ee5-0738-4551-b4a7-631f4f0339aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=21d050a2697746719c75641f70da0cfd, user_id=f393f3fac09d40aa87290de0f46040b2, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:32.850383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a3ee5-0738-4551-b4a7-631f4f0339aa admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:32.935660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 999/2029] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:32 2022] PUT /identity/v3/projects/21d050a2697746719c75641f70da0cfd/users/f393f3fac09d40aa87290de0f46040b2/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:32.949259 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc8d3e-853d-48b3-a840-4dd705697adb None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:32.949494 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc8d3e-853d-48b3-a840-4dd705697adb None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:32.949747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc8d3e-853d-48b3-a840-4dd705697adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:32.981363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-bcbc8d3e-853d-48b3-a840-4dd705697adb None None] MFA Rules not processed for user `f393f3fac09d40aa87290de0f46040b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:32.982629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bcbc8d3e-853d-48b3-a840-4dd705697adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:33.064053 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1031/2030] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:33.208008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a3ee5-0738-4551-b4a7-631f4f0339aa admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:33.209988 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-409a3ee5-0738-4551-b4a7-631f4f0339aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:33.302125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a3ee5-0738-4551-b4a7-631f4f0339aa admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:33.306719 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692f41e-7883-46e4-8e1e-da22d44c670f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:33.307655 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692f41e-7883-46e4-8e1e-da22d44c670f service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:33.307958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692f41e-7883-46e4-8e1e-da22d44c670f service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:33.308245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692f41e-7883-46e4-8e1e-da22d44c670f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:33.323353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2692f41e-7883-46e4-8e1e-da22d44c670f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:33.325415 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2692f41e-7883-46e4-8e1e-da22d44c670f service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:33.349347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1000/2031] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:43.059822 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=49,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:43.333340 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=57,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:43.828406 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc8d3e-853d-48b3-a840-4dd705697adb None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:43.899990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc8d3e-853d-48b3-a840-4dd705697adb None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:43.905641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b749eac6-724d-4622-9401-a48b3852ff52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:43.906566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749eac6-724d-4622-9401-a48b3852ff52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:43.907361 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749eac6-724d-4622-9401-a48b3852ff52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:43.907645 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749eac6-724d-4622-9401-a48b3852ff52 admin admin] PATH_INFO: `/v3/users/f393f3fac09d40aa87290de0f46040b2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:43.908995 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b749eac6-724d-4622-9401-a48b3852ff52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f393f3fac09d40aa87290de0f46040b2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:43.911840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b749eac6-724d-4622-9401-a48b3852ff52 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:44.134389 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b749eac6-724d-4622-9401-a48b3852ff52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f393f3fac09d40aa87290de0f46040b2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:44.135133 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b749eac6-724d-4622-9401-a48b3852ff52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f393f3fac09d40aa87290de0f46040b2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:44.151276 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b749eac6-724d-4622-9401-a48b3852ff52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f393f3fac09d40aa87290de0f46040b2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:44.152039 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b749eac6-724d-4622-9401-a48b3852ff52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f393f3fac09d40aa87290de0f46040b2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:44.162909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b749eac6-724d-4622-9401-a48b3852ff52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f393f3fac09d40aa87290de0f46040b2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:44.270100 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1032/2032] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:43 2022] DELETE /identity/v3/users/f393f3fac09d40aa87290de0f46040b2 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:44.275946 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95644193-7c6c-4c53-a077-c592f1ada2ae None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:44.278734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95644193-7c6c-4c53-a077-c592f1ada2ae None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:44.279506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95644193-7c6c-4c53-a077-c592f1ada2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:44.316301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-95644193-7c6c-4c53-a077-c592f1ada2ae None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:44.317663 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-95644193-7c6c-4c53-a077-c592f1ada2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:44.413962 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1001/2033] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:44.424351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b749eac6-724d-4622-9401-a48b3852ff52 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:44.425950 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b749eac6-724d-4622-9401-a48b3852ff52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:44.501506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b749eac6-724d-4622-9401-a48b3852ff52 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:44.505506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fddd9-f635-47b1-8cc2-a4e248ca6741 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:44.506407 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fddd9-f635-47b1-8cc2-a4e248ca6741 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:44.506747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fddd9-f635-47b1-8cc2-a4e248ca6741 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:44.507095 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fddd9-f635-47b1-8cc2-a4e248ca6741 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:44.517504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fddd9-f635-47b1-8cc2-a4e248ca6741 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:44.518963 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fddd9-f635-47b1-8cc2-a4e248ca6741 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:44.529794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1033/2034] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:45.117370 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95644193-7c6c-4c53-a077-c592f1ada2ae None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:45.124396 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-95644193-7c6c-4c53-a077-c592f1ada2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:45.132091 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95644193-7c6c-4c53-a077-c592f1ada2ae None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:45.138586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427c855-c34e-4e5a-bd80-91be722609e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:45.139507 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427c855-c34e-4e5a-bd80-91be722609e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:45.139798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427c855-c34e-4e5a-bd80-91be722609e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:45.140072 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427c855-c34e-4e5a-bd80-91be722609e2 admin admin] PATH_INFO: `/v3/projects/21d050a2697746719c75641f70da0cfd` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:45.141316 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a427c855-c34e-4e5a-bd80-91be722609e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21d050a2697746719c75641f70da0cfd)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:45.143319 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a427c855-c34e-4e5a-bd80-91be722609e2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:45.282565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a427c855-c34e-4e5a-bd80-91be722609e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21d050a2697746719c75641f70da0cfd', 'request_id': 'req-a427c855-c34e-4e5a-bd80-91be722609e2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:45.366872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a427c855-c34e-4e5a-bd80-91be722609e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21d050a2697746719c75641f70da0cfd', 'request_id': 'req-a427c855-c34e-4e5a-bd80-91be722609e2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:45.375683 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a427c855-c34e-4e5a-bd80-91be722609e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21d050a2697746719c75641f70da0cfd', 'request_id': 'req-a427c855-c34e-4e5a-bd80-91be722609e2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:45.392846 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a427c855-c34e-4e5a-bd80-91be722609e2 admin admin] The token cache is being invalidate because project 21d050a2697746719c75641f70da0cfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:45.393205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a427c855-c34e-4e5a-bd80-91be722609e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:45.393947 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a427c855-c34e-4e5a-bd80-91be722609e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:45.401504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1002/2035] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:45 2022] DELETE /identity/v3/projects/21d050a2697746719c75641f70da0cfd => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:45.403447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57003e7e-c11f-49e3-b59a-c5d8b1326139 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:45.403899 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57003e7e-c11f-49e3-b59a-c5d8b1326139 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:45.404082 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57003e7e-c11f-49e3-b59a-c5d8b1326139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:45.424168 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-57003e7e-c11f-49e3-b59a-c5d8b1326139 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:45.425413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-57003e7e-c11f-49e3-b59a-c5d8b1326139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:45.451526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:45.513905 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1034/2036] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:45.529535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ebc5d-658f-4706-b89c-ba21b7012c4e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:45.538256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ebc5d-658f-4706-b89c-ba21b7012c4e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:45.538256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ebc5d-658f-4706-b89c-ba21b7012c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:45.558822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-865ebc5d-658f-4706-b89c-ba21b7012c4e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:45.560848 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-865ebc5d-658f-4706-b89c-ba21b7012c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:45.576903 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1003/2037] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:45.580237 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57003e7e-c11f-49e3-b59a-c5d8b1326139 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:45.602175 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57003e7e-c11f-49e3-b59a-c5d8b1326139 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:45.609105 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c7188-3c13-456d-9557-77a2d23d936d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:45.610050 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c7188-3c13-456d-9557-77a2d23d936d admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:45.610609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c7188-3c13-456d-9557-77a2d23d936d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:45.611073 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c7188-3c13-456d-9557-77a2d23d936d admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:45.612090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c7188-3c13-456d-9557-77a2d23d936d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:09:45.619101 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c7188-3c13-456d-9557-77a2d23d936d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:45.619101 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c7188-3c13-456d-9557-77a2d23d936d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:45.631715 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1035/2038] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:09:45.635795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ebc5d-658f-4706-b89c-ba21b7012c4e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:45.647868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ebc5d-658f-4706-b89c-ba21b7012c4e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:45.652847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937331e6-ddaa-4bb6-969f-580941639f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:45.653669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937331e6-ddaa-4bb6-969f-580941639f4f admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:45.654202 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937331e6-ddaa-4bb6-969f-580941639f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:45.655097 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937331e6-ddaa-4bb6-969f-580941639f4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:45.655304 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937331e6-ddaa-4bb6-969f-580941639f4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:45.657228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937331e6-ddaa-4bb6-969f-580941639f4f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:45.788516 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1004/2039] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:45 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:09:45.789271 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c7188-3c13-456d-9557-77a2d23d936d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:45.800030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c7188-3c13-456d-9557-77a2d23d936d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:45.878796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa907530-8f2e-4156-af58-55747927be79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:45.879593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa907530-8f2e-4156-af58-55747927be79 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:45.879814 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa907530-8f2e-4156-af58-55747927be79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:45.880100 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa907530-8f2e-4156-af58-55747927be79 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:45.898161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa907530-8f2e-4156-af58-55747927be79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:45.898161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa907530-8f2e-4156-af58-55747927be79 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:46.042826 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1036/2040] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:45 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:09:46.048525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937331e6-ddaa-4bb6-969f-580941639f4f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:46.059689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937331e6-ddaa-4bb6-969f-580941639f4f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:46.065211 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960afded-8982-47dd-b43a-f558027a2b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:46.065898 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960afded-8982-47dd-b43a-f558027a2b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:46.066206 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960afded-8982-47dd-b43a-f558027a2b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:46.066509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960afded-8982-47dd-b43a-f558027a2b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:46.068109 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960afded-8982-47dd-b43a-f558027a2b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:46.069022 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960afded-8982-47dd-b43a-f558027a2b64 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:46.083842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1005/2041] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:09:46.088024 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa907530-8f2e-4156-af58-55747927be79 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:46.098513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa907530-8f2e-4156-af58-55747927be79 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:46.103294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e12d2-dc2d-40c8-bef6-066ade563d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:46.103966 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e12d2-dc2d-40c8-bef6-066ade563d10 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:46.104354 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e12d2-dc2d-40c8-bef6-066ade563d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:46.104637 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e12d2-dc2d-40c8-bef6-066ade563d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:46.105368 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e12d2-dc2d-40c8-bef6-066ade563d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:46.107055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e12d2-dc2d-40c8-bef6-066ade563d10 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:46.117828 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1037/2042] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:46.128568 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960afded-8982-47dd-b43a-f558027a2b64 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:46.139744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960afded-8982-47dd-b43a-f558027a2b64 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:46.144622 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571e917-0307-48dd-a171-a64f1ad5a807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:46.145518 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571e917-0307-48dd-a171-a64f1ad5a807 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:46.145808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571e917-0307-48dd-a171-a64f1ad5a807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:46.146090 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571e917-0307-48dd-a171-a64f1ad5a807 admin admin] PATH_INFO: `/v3/projects/0bf5ec66c66f4d9e9ed5aafc2d799350/users/f67414953d2b432987e74a6d00d824f4/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:46.163248 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7571e917-0307-48dd-a171-a64f1ad5a807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bf5ec66c66f4d9e9ed5aafc2d799350, user_id=f67414953d2b432987e74a6d00d824f4, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:46.165927 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7571e917-0307-48dd-a171-a64f1ad5a807 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:46.266542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1006/2043] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:46 2022] PUT /identity/v3/projects/0bf5ec66c66f4d9e9ed5aafc2d799350/users/f67414953d2b432987e74a6d00d824f4/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:46.278683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f01c3-4e99-4a27-9c48-06a3919b9a79 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:46.278982 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f01c3-4e99-4a27-9c48-06a3919b9a79 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:46.279300 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f01c3-4e99-4a27-9c48-06a3919b9a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:46.316010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-fc0f01c3-4e99-4a27-9c48-06a3919b9a79 None None] MFA Rules not processed for user `f67414953d2b432987e74a6d00d824f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:46.317669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-fc0f01c3-4e99-4a27-9c48-06a3919b9a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:46.403381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1038/2044] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:46.408105 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571e917-0307-48dd-a171-a64f1ad5a807 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:46.436362 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:46.494829 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571e917-0307-48dd-a171-a64f1ad5a807 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:46.508369 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81251d13-bb89-4921-ad4e-45b7b27f0438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:46.508944 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81251d13-bb89-4921-ad4e-45b7b27f0438 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:46.509233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81251d13-bb89-4921-ad4e-45b7b27f0438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:46.509553 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81251d13-bb89-4921-ad4e-45b7b27f0438 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:46.510208 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81251d13-bb89-4921-ad4e-45b7b27f0438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:46.512863 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81251d13-bb89-4921-ad4e-45b7b27f0438 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:46.694076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1007/2045] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:46 2022] POST /identity/v3/projects => generated 385 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:46.698303 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f01c3-4e99-4a27-9c48-06a3919b9a79 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:46.710894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f01c3-4e99-4a27-9c48-06a3919b9a79 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:46.726100 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:46.779592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f6299-7010-4f97-946b-1015596cd205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:46.780286 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f6299-7010-4f97-946b-1015596cd205 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:46.780578 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f6299-7010-4f97-946b-1015596cd205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:46.780860 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f6299-7010-4f97-946b-1015596cd205 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:46.781561 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f6299-7010-4f97-946b-1015596cd205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:46.783845 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f6299-7010-4f97-946b-1015596cd205 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:46.880162 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1039/2046] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:46 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:46.888495 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81251d13-bb89-4921-ad4e-45b7b27f0438 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:46.904653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81251d13-bb89-4921-ad4e-45b7b27f0438 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:46.909472 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f747608-cc18-41bd-8a48-92c789c7fb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:46.910299 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f747608-cc18-41bd-8a48-92c789c7fb61 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:46.910591 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f747608-cc18-41bd-8a48-92c789c7fb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:46.910927 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f747608-cc18-41bd-8a48-92c789c7fb61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:46.911696 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f747608-cc18-41bd-8a48-92c789c7fb61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:46.913321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f747608-cc18-41bd-8a48-92c789c7fb61 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:46.928730 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1008/2047] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:46.933628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f6299-7010-4f97-946b-1015596cd205 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:46.942458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f6299-7010-4f97-946b-1015596cd205 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:46.956198 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83269c-4f1b-4b91-baf5-c32d520bd0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:46.956198 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83269c-4f1b-4b91-baf5-c32d520bd0da admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:46.956198 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83269c-4f1b-4b91-baf5-c32d520bd0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:46.956198 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83269c-4f1b-4b91-baf5-c32d520bd0da admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:46.956549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83269c-4f1b-4b91-baf5-c32d520bd0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:46.956549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83269c-4f1b-4b91-baf5-c32d520bd0da admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:46.966363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1040/2048] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:46.970727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f747608-cc18-41bd-8a48-92c789c7fb61 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:46.982054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f747608-cc18-41bd-8a48-92c789c7fb61 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:46.987343 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef634e-cd35-4f6f-aac0-14d449717ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:46.988480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef634e-cd35-4f6f-aac0-14d449717ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:46.988568 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef634e-cd35-4f6f-aac0-14d449717ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:46.988895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef634e-cd35-4f6f-aac0-14d449717ac8 admin admin] PATH_INFO: `/v3/projects/9fd7fdefaeed4c8886a69652ac419c61/users/1f0dad75217e46589df50e1d7a644d75/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:47.018448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aef634e-cd35-4f6f-aac0-14d449717ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fd7fdefaeed4c8886a69652ac419c61, user_id=1f0dad75217e46589df50e1d7a644d75, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:47.021307 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aef634e-cd35-4f6f-aac0-14d449717ac8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:47.110484 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1009/2049] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:46 2022] PUT /identity/v3/projects/9fd7fdefaeed4c8886a69652ac419c61/users/1f0dad75217e46589df50e1d7a644d75/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:47.121695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d2d4f-b583-4aa3-90ca-4334442004c8 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:47.121989 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d2d4f-b583-4aa3-90ca-4334442004c8 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:47.122330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d2d4f-b583-4aa3-90ca-4334442004c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:47.151512 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-867d2d4f-b583-4aa3-90ca-4334442004c8 None None] MFA Rules not processed for user `1f0dad75217e46589df50e1d7a644d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:47.152844 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-867d2d4f-b583-4aa3-90ca-4334442004c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:47.233560 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1041/2050] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:47.238139 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef634e-cd35-4f6f-aac0-14d449717ac8 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:47.317573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef634e-cd35-4f6f-aac0-14d449717ac8 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:47.325379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fb427-0e4f-4f65-b907-450f2af80131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:47.326068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fb427-0e4f-4f65-b907-450f2af80131 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:47.328509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fb427-0e4f-4f65-b907-450f2af80131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:47.328509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fb427-0e4f-4f65-b907-450f2af80131 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:47.328509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fb427-0e4f-4f65-b907-450f2af80131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:47.329344 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fb427-0e4f-4f65-b907-450f2af80131 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:47.448668 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1010/2051] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:47 2022] POST /identity/v3/projects => generated 385 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:47.462647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d2d4f-b583-4aa3-90ca-4334442004c8 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:47.475278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d2d4f-b583-4aa3-90ca-4334442004c8 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:47.546330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56660907-855c-4902-b8b7-94caa360a1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:47.547022 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56660907-855c-4902-b8b7-94caa360a1a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:47.547453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56660907-855c-4902-b8b7-94caa360a1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:47.548027 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56660907-855c-4902-b8b7-94caa360a1a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:47.548839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56660907-855c-4902-b8b7-94caa360a1a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:47.550742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56660907-855c-4902-b8b7-94caa360a1a0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:47.644152 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1042/2052] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:47 2022] POST /identity/v3/users => generated 299 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:47.648541 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fb427-0e4f-4f65-b907-450f2af80131 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:47.660901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fb427-0e4f-4f65-b907-450f2af80131 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:47.667954 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1479bcc-2aff-4c2f-b1bc-4e46517bf2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:47.669794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1479bcc-2aff-4c2f-b1bc-4e46517bf2cc admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:47.670390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1479bcc-2aff-4c2f-b1bc-4e46517bf2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:47.670828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1479bcc-2aff-4c2f-b1bc-4e46517bf2cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:47.671969 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1479bcc-2aff-4c2f-b1bc-4e46517bf2cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:47.674125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1479bcc-2aff-4c2f-b1bc-4e46517bf2cc admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:47.691986 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1011/2053] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:47.693384 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56660907-855c-4902-b8b7-94caa360a1a0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:47.706329 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56660907-855c-4902-b8b7-94caa360a1a0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:47.724020 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1004ce-d3a8-4852-981f-a6dc03d3fe36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:47.731952 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1004ce-d3a8-4852-981f-a6dc03d3fe36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:47.733997 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1004ce-d3a8-4852-981f-a6dc03d3fe36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:47.734736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1004ce-d3a8-4852-981f-a6dc03d3fe36 admin admin] PATH_INFO: `/v3/projects/3bfda1a3a0ae40c1a1cbe82570740227/users/4fd3c1b274534d9395d8d72a75591aa3/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:47.761189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:47.766579 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1004ce-d3a8-4852-981f-a6dc03d3fe36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bfda1a3a0ae40c1a1cbe82570740227, user_id=4fd3c1b274534d9395d8d72a75591aa3, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:47.774679 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1004ce-d3a8-4852-981f-a6dc03d3fe36 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:47.845680 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1043/2054] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:47 2022] PUT /identity/v3/projects/3bfda1a3a0ae40c1a1cbe82570740227/users/4fd3c1b274534d9395d8d72a75591aa3/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:47.850642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1479bcc-2aff-4c2f-b1bc-4e46517bf2cc admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:47.865373 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1479bcc-2aff-4c2f-b1bc-4e46517bf2cc admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:47.911218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:47.928956 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ef411-7dd2-4d29-81cd-bd94efc92cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:47.929650 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ef411-7dd2-4d29-81cd-bd94efc92cbe admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:47.929940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ef411-7dd2-4d29-81cd-bd94efc92cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:47.930233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ef411-7dd2-4d29-81cd-bd94efc92cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:47.930961 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ef411-7dd2-4d29-81cd-bd94efc92cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:47.932687 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ef411-7dd2-4d29-81cd-bd94efc92cbe admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:47.947916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1012/2055] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:09:47.949160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1004ce-d3a8-4852-981f-a6dc03d3fe36 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:47.960467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1004ce-d3a8-4852-981f-a6dc03d3fe36 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:47.966304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2a5fd-faef-4732-a9b8-8deebb3ef9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:47.967079 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2a5fd-faef-4732-a9b8-8deebb3ef9df admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:47.967401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2a5fd-faef-4732-a9b8-8deebb3ef9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:47.967684 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2a5fd-faef-4732-a9b8-8deebb3ef9df admin admin] PATH_INFO: `/v3/projects/3bfda1a3a0ae40c1a1cbe82570740227/users/4fd3c1b274534d9395d8d72a75591aa3/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:47.969715 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2a5fd-faef-4732-a9b8-8deebb3ef9df admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bfda1a3a0ae40c1a1cbe82570740227, user_id=4fd3c1b274534d9395d8d72a75591aa3, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:47.972440 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2a5fd-faef-4732-a9b8-8deebb3ef9df admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:47.987709 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1044/2056] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:47 2022] PUT /identity/v3/projects/3bfda1a3a0ae40c1a1cbe82570740227/users/4fd3c1b274534d9395d8d72a75591aa3/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:47.999536 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fd73e-7830-4868-b67e-faba5ce2e53c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:47.999739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fd73e-7830-4868-b67e-faba5ce2e53c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:47.999960 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fd73e-7830-4868-b67e-faba5ce2e53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:48.031627 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4c9fd73e-7830-4868-b67e-faba5ce2e53c None None] MFA Rules not processed for user `4fd3c1b274534d9395d8d72a75591aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:48.032870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4c9fd73e-7830-4868-b67e-faba5ce2e53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:48.105666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1013/2057] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:48.117229 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2a5fd-faef-4732-a9b8-8deebb3ef9df admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:48.118943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e3d2a5fd-faef-4732-a9b8-8deebb3ef9df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:48.168078 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fd73e-7830-4868-b67e-faba5ce2e53c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:48.169268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4c9fd73e-7830-4868-b67e-faba5ce2e53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:48.185061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2a5fd-faef-4732-a9b8-8deebb3ef9df admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:48.189937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e1f60-8e8a-4caf-bbef-39183af24036 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:48.190609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e1f60-8e8a-4caf-bbef-39183af24036 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:48.190905 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e1f60-8e8a-4caf-bbef-39183af24036 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:48.191235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e1f60-8e8a-4caf-bbef-39183af24036 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:48.240687 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fd73e-7830-4868-b67e-faba5ce2e53c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:48.246508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc0b2d-de60-4d54-a910-2cbbdd7bc8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:48.247437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc0b2d-de60-4d54-a910-2cbbdd7bc8ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:48.247728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc0b2d-de60-4d54-a910-2cbbdd7bc8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:48.248011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc0b2d-de60-4d54-a910-2cbbdd7bc8ef admin admin] PATH_INFO: `/v3/users/cdd54a12768648ceb46e68bbe626606f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:48.249339 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc0b2d-de60-4d54-a910-2cbbdd7bc8ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdd54a12768648ceb46e68bbe626606f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:48.251386 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc0b2d-de60-4d54-a910-2cbbdd7bc8ef admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:48.270895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e1f60-8e8a-4caf-bbef-39183af24036 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:48.272695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e1f60-8e8a-4caf-bbef-39183af24036 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:48.283479 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1045/2058] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:48.520820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-aadc0b2d-de60-4d54-a910-2cbbdd7bc8ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdd54a12768648ceb46e68bbe626606f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:48.529195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-aadc0b2d-de60-4d54-a910-2cbbdd7bc8ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdd54a12768648ceb46e68bbe626606f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:48.530758 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-aadc0b2d-de60-4d54-a910-2cbbdd7bc8ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdd54a12768648ceb46e68bbe626606f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:48.530758 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-aadc0b2d-de60-4d54-a910-2cbbdd7bc8ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdd54a12768648ceb46e68bbe626606f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:48.547735 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:48.552632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-aadc0b2d-de60-4d54-a910-2cbbdd7bc8ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdd54a12768648ceb46e68bbe626606f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:48.660579 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1014/2059] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:48 2022] DELETE /identity/v3/users/cdd54a12768648ceb46e68bbe626606f => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:48.665358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70899e-bd82-43a4-8d44-c68b0c276bbc None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:48.665655 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70899e-bd82-43a4-8d44-c68b0c276bbc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:48.665970 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70899e-bd82-43a4-8d44-c68b0c276bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:48.685140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0c70899e-bd82-43a4-8d44-c68b0c276bbc None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:48.686430 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0c70899e-bd82-43a4-8d44-c68b0c276bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:48.749243 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1046/2060] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:09:48.761121 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc0b2d-de60-4d54-a910-2cbbdd7bc8ef admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:48.763001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-aadc0b2d-de60-4d54-a910-2cbbdd7bc8ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:48.832916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc0b2d-de60-4d54-a910-2cbbdd7bc8ef admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:48.836921 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48666ecd-873e-4003-8d05-109519631774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:48.837583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48666ecd-873e-4003-8d05-109519631774 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:48.837870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48666ecd-873e-4003-8d05-109519631774 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:48.838148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48666ecd-873e-4003-8d05-109519631774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:48.848634 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48666ecd-873e-4003-8d05-109519631774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:48.850086 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48666ecd-873e-4003-8d05-109519631774 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:48.863927 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1015/2061] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:49.479072 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70899e-bd82-43a4-8d44-c68b0c276bbc None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:49.480720 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0c70899e-bd82-43a4-8d44-c68b0c276bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:49.498363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70899e-bd82-43a4-8d44-c68b0c276bbc None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:49.503963 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baceba84-a9bb-4d68-8749-f730443f7d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:49.504893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baceba84-a9bb-4d68-8749-f730443f7d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:49.505261 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baceba84-a9bb-4d68-8749-f730443f7d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:49.505614 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baceba84-a9bb-4d68-8749-f730443f7d12 admin admin] PATH_INFO: `/v3/projects/cc29a1f5e0054fc0976a1059f0fb760b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:49.507039 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baceba84-a9bb-4d68-8749-f730443f7d12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc29a1f5e0054fc0976a1059f0fb760b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:49.510533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baceba84-a9bb-4d68-8749-f730443f7d12 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:49.614557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:49.635954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-baceba84-a9bb-4d68-8749-f730443f7d12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc29a1f5e0054fc0976a1059f0fb760b', 'request_id': 'req-baceba84-a9bb-4d68-8749-f730443f7d12'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:49.727809 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-baceba84-a9bb-4d68-8749-f730443f7d12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc29a1f5e0054fc0976a1059f0fb760b', 'request_id': 'req-baceba84-a9bb-4d68-8749-f730443f7d12'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:49.735496 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-baceba84-a9bb-4d68-8749-f730443f7d12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc29a1f5e0054fc0976a1059f0fb760b', 'request_id': 'req-baceba84-a9bb-4d68-8749-f730443f7d12'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:49.758493 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-baceba84-a9bb-4d68-8749-f730443f7d12 admin admin] The token cache is being invalidate because project cc29a1f5e0054fc0976a1059f0fb760b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:49.759543 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-baceba84-a9bb-4d68-8749-f730443f7d12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:49.760588 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-baceba84-a9bb-4d68-8749-f730443f7d12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:49.763278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1047/2062] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:49 2022] DELETE /identity/v3/projects/cc29a1f5e0054fc0976a1059f0fb760b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:49.767373 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48666ecd-873e-4003-8d05-109519631774 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:49.777604 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48666ecd-873e-4003-8d05-109519631774 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:49.850297 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff2e32-f1e3-4cc4-9348-ce6448316e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:49.851305 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff2e32-f1e3-4cc4-9348-ce6448316e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:49.851600 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff2e32-f1e3-4cc4-9348-ce6448316e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:49.851904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff2e32-f1e3-4cc4-9348-ce6448316e81 admin admin] PATH_INFO: `/v3/users/d780c67ddb274103afe60e7a8837390d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:49.853500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff2e32-f1e3-4cc4-9348-ce6448316e81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d780c67ddb274103afe60e7a8837390d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:49.856861 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff2e32-f1e3-4cc4-9348-ce6448316e81 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:50.121459 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-edff2e32-f1e3-4cc4-9348-ce6448316e81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd780c67ddb274103afe60e7a8837390d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:50.131114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-edff2e32-f1e3-4cc4-9348-ce6448316e81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd780c67ddb274103afe60e7a8837390d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:50.131931 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-edff2e32-f1e3-4cc4-9348-ce6448316e81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd780c67ddb274103afe60e7a8837390d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:50.132596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-edff2e32-f1e3-4cc4-9348-ce6448316e81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd780c67ddb274103afe60e7a8837390d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:50.170222 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-edff2e32-f1e3-4cc4-9348-ce6448316e81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd780c67ddb274103afe60e7a8837390d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:50.230121 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1016/2063] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:49 2022] DELETE /identity/v3/users/d780c67ddb274103afe60e7a8837390d => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:50.677230 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baceba84-a9bb-4d68-8749-f730443f7d12 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:50.680763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-baceba84-a9bb-4d68-8749-f730443f7d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:50.702986 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baceba84-a9bb-4d68-8749-f730443f7d12 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:50.782330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab4a94-a16c-4c6c-9a84-6a6364e7c7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:50.783064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab4a94-a16c-4c6c-9a84-6a6364e7c7b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:50.783285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab4a94-a16c-4c6c-9a84-6a6364e7c7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:50.783584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab4a94-a16c-4c6c-9a84-6a6364e7c7b7 admin admin] PATH_INFO: `/v3/projects/7cb97d9c20294d9caeb09b5ecc111b87` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:50.784886 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab4a94-a16c-4c6c-9a84-6a6364e7c7b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cb97d9c20294d9caeb09b5ecc111b87)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:50.787001 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab4a94-a16c-4c6c-9a84-6a6364e7c7b7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:50.931317 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-29ab4a94-a16c-4c6c-9a84-6a6364e7c7b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cb97d9c20294d9caeb09b5ecc111b87', 'request_id': 'req-29ab4a94-a16c-4c6c-9a84-6a6364e7c7b7'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:51.041656 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-29ab4a94-a16c-4c6c-9a84-6a6364e7c7b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cb97d9c20294d9caeb09b5ecc111b87', 'request_id': 'req-29ab4a94-a16c-4c6c-9a84-6a6364e7c7b7'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:51.049083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-29ab4a94-a16c-4c6c-9a84-6a6364e7c7b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cb97d9c20294d9caeb09b5ecc111b87', 'request_id': 'req-29ab4a94-a16c-4c6c-9a84-6a6364e7c7b7'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:51.061425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-29ab4a94-a16c-4c6c-9a84-6a6364e7c7b7 admin admin] The token cache is being invalidate because project 7cb97d9c20294d9caeb09b5ecc111b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:51.061714 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-29ab4a94-a16c-4c6c-9a84-6a6364e7c7b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:51.062369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-29ab4a94-a16c-4c6c-9a84-6a6364e7c7b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:51.064047 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1048/2064] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:50 2022] DELETE /identity/v3/projects/7cb97d9c20294d9caeb09b5ecc111b87 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:51.069602 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53a592-b8b1-469b-b230-7b12509f57f5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:51.070002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53a592-b8b1-469b-b230-7b12509f57f5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:51.070520 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53a592-b8b1-469b-b230-7b12509f57f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:51.084868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:51.092042 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3a53a592-b8b1-469b-b230-7b12509f57f5 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:51.093314 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3a53a592-b8b1-469b-b230-7b12509f57f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:51.159375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1017/2065] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:09:51.169518 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96a139-d880-4c08-adb5-436616fc6515 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:51.169908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96a139-d880-4c08-adb5-436616fc6515 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:51.170262 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96a139-d880-4c08-adb5-436616fc6515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:51.191490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-aa96a139-d880-4c08-adb5-436616fc6515 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:51.192795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-aa96a139-d880-4c08-adb5-436616fc6515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:51.211341 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1049/2066] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:51.212155 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53a592-b8b1-469b-b230-7b12509f57f5 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:51.230492 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53a592-b8b1-469b-b230-7b12509f57f5 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:51.232178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c0454-efe9-4843-8bb3-4f23d73e301b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:51.232876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c0454-efe9-4843-8bb3-4f23d73e301b admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:51.233334 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c0454-efe9-4843-8bb3-4f23d73e301b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:51.233710 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c0454-efe9-4843-8bb3-4f23d73e301b admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:51.234524 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c0454-efe9-4843-8bb3-4f23d73e301b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:09:51.234836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c0454-efe9-4843-8bb3-4f23d73e301b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:51.236619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c0454-efe9-4843-8bb3-4f23d73e301b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:51.258119 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1018/2067] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:09:51.259082 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96a139-d880-4c08-adb5-436616fc6515 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:51.269097 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96a139-d880-4c08-adb5-436616fc6515 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:51.274983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59e43f-889b-4ec1-b025-da20ff9732c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:51.275705 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59e43f-889b-4ec1-b025-da20ff9732c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:51.275993 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59e43f-889b-4ec1-b025-da20ff9732c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:51.276269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59e43f-889b-4ec1-b025-da20ff9732c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:51.277045 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59e43f-889b-4ec1-b025-da20ff9732c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:51.287518 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59e43f-889b-4ec1-b025-da20ff9732c7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:51.390807 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c0454-efe9-4843-8bb3-4f23d73e301b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:51.391044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1050/2068] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:51 2022] POST /identity/v3/projects => generated 363 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:51.406617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c0454-efe9-4843-8bb3-4f23d73e301b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:51.501848 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f10b05-f7d6-423d-bc50-063d0af21fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:51.502606 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f10b05-f7d6-423d-bc50-063d0af21fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:51.502898 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f10b05-f7d6-423d-bc50-063d0af21fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:51.503199 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f10b05-f7d6-423d-bc50-063d0af21fd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:51.504000 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f10b05-f7d6-423d-bc50-063d0af21fd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:51.506145 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f10b05-f7d6-423d-bc50-063d0af21fd6 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:51.588987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59e43f-889b-4ec1-b025-da20ff9732c7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:51.592569 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4c59e43f-889b-4ec1-b025-da20ff9732c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:51.617910 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:51.630797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1019/2069] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:51 2022] POST /identity/v3/users => generated 282 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:51.635397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f10b05-f7d6-423d-bc50-063d0af21fd6 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:51.650142 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f10b05-f7d6-423d-bc50-063d0af21fd6 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:51.656747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35912e8-fd46-4da6-8b64-3b5b44a86081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:51.664734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35912e8-fd46-4da6-8b64-3b5b44a86081 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:51.665387 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35912e8-fd46-4da6-8b64-3b5b44a86081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:51.665582 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35912e8-fd46-4da6-8b64-3b5b44a86081 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:51.666527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35912e8-fd46-4da6-8b64-3b5b44a86081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:51.670347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35912e8-fd46-4da6-8b64-3b5b44a86081 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:51.685390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1020/2070] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:09:51.685883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59e43f-889b-4ec1-b025-da20ff9732c7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:51.686347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35912e8-fd46-4da6-8b64-3b5b44a86081 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:51.691299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4167f-20e1-42c4-a880-0b40972ea329 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:51.692209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4167f-20e1-42c4-a880-0b40972ea329 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:51.692505 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4167f-20e1-42c4-a880-0b40972ea329 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:51.692789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4167f-20e1-42c4-a880-0b40972ea329 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:51.694797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cff4167f-20e1-42c4-a880-0b40972ea329 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:51.697598 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35912e8-fd46-4da6-8b64-3b5b44a86081 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:51.711548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad38d8-6ff8-427c-9fc7-9780788cfa92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:51.712253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad38d8-6ff8-427c-9fc7-9780788cfa92 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:51.712654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad38d8-6ff8-427c-9fc7-9780788cfa92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:51.712813 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad38d8-6ff8-427c-9fc7-9780788cfa92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:51.713731 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad38d8-6ff8-427c-9fc7-9780788cfa92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:51.716131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad38d8-6ff8-427c-9fc7-9780788cfa92 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:51.733492 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1021/2071] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:51.737907 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad38d8-6ff8-427c-9fc7-9780788cfa92 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:51.764616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad38d8-6ff8-427c-9fc7-9780788cfa92 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:51.778587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e787c16-909a-4ea6-ac64-c6a8ef1d7903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:51.779451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e787c16-909a-4ea6-ac64-c6a8ef1d7903 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:51.779890 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e787c16-909a-4ea6-ac64-c6a8ef1d7903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:51.780182 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e787c16-909a-4ea6-ac64-c6a8ef1d7903 admin admin] PATH_INFO: `/v3/projects/f4219e8f4ef34bf9a89a313b2e044f67/users/c7d4274811554f27a25a9ce786d7f802/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:51.816556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e787c16-909a-4ea6-ac64-c6a8ef1d7903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4219e8f4ef34bf9a89a313b2e044f67, user_id=c7d4274811554f27a25a9ce786d7f802, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:51.820108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e787c16-909a-4ea6-ac64-c6a8ef1d7903 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:51.829833 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4167f-20e1-42c4-a880-0b40972ea329 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:51.831520 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4167f-20e1-42c4-a880-0b40972ea329 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:51.843546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1051/2072] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:09:51.919806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1022/2073] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:51 2022] PUT /identity/v3/projects/f4219e8f4ef34bf9a89a313b2e044f67/users/c7d4274811554f27a25a9ce786d7f802/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:51.929652 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6261cd-597f-41bb-9f1e-5ab090515505 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:51.930132 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6261cd-597f-41bb-9f1e-5ab090515505 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:51.930636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6261cd-597f-41bb-9f1e-5ab090515505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:51.988363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5b6261cd-597f-41bb-9f1e-5ab090515505 None None] MFA Rules not processed for user `c7d4274811554f27a25a9ce786d7f802`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:51.991805 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5b6261cd-597f-41bb-9f1e-5ab090515505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:51.997538 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e787c16-909a-4ea6-ac64-c6a8ef1d7903 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:51.999858 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1e787c16-909a-4ea6-ac64-c6a8ef1d7903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:52.122869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1052/2074] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:51 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:52.123830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6261cd-597f-41bb-9f1e-5ab090515505 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:52.126319 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:52.162469 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e787c16-909a-4ea6-ac64-c6a8ef1d7903 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:52.177962 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22dd581-ab50-4dcc-a858-45be739bd401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:52.179410 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dd581-ab50-4dcc-a858-45be739bd401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:52.180019 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dd581-ab50-4dcc-a858-45be739bd401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:52.180448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dd581-ab50-4dcc-a858-45be739bd401 admin admin] PATH_INFO: `/v3/users/b6168a02789945d68a93c0c06d785753` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:52.182101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22dd581-ab50-4dcc-a858-45be739bd401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6168a02789945d68a93c0c06d785753)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:52.186902 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22dd581-ab50-4dcc-a858-45be739bd401 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:52.225898 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6261cd-597f-41bb-9f1e-5ab090515505 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:52.232113 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5c4aa-22c9-4426-86b9-c7e47078154f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:52.232792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5c4aa-22c9-4426-86b9-c7e47078154f admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:52.233137 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5c4aa-22c9-4426-86b9-c7e47078154f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:52.233422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5c4aa-22c9-4426-86b9-c7e47078154f admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:52.234219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5c4aa-22c9-4426-86b9-c7e47078154f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:52.236321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5c4aa-22c9-4426-86b9-c7e47078154f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:52.241272 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:52.360916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1053/2075] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:52 2022] POST /identity/v3/projects => generated 363 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:52.362157 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5c4aa-22c9-4426-86b9-c7e47078154f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:52.402711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c22dd581-ab50-4dcc-a858-45be739bd401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6168a02789945d68a93c0c06d785753'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:52.419214 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c22dd581-ab50-4dcc-a858-45be739bd401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6168a02789945d68a93c0c06d785753'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:52.421340 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c22dd581-ab50-4dcc-a858-45be739bd401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6168a02789945d68a93c0c06d785753'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:52.422267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c22dd581-ab50-4dcc-a858-45be739bd401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6168a02789945d68a93c0c06d785753'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:52.443880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c22dd581-ab50-4dcc-a858-45be739bd401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6168a02789945d68a93c0c06d785753'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:52.481274 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5c4aa-22c9-4426-86b9-c7e47078154f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:52.488799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43b202-9cea-47e7-822d-76be3f6fc56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:52.489687 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43b202-9cea-47e7-822d-76be3f6fc56a admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:52.490075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43b202-9cea-47e7-822d-76be3f6fc56a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:52.490462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43b202-9cea-47e7-822d-76be3f6fc56a admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:52.491298 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43b202-9cea-47e7-822d-76be3f6fc56a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:52.493392 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43b202-9cea-47e7-822d-76be3f6fc56a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:52.562615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1023/2076] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:51 2022] DELETE /identity/v3/users/b6168a02789945d68a93c0c06d785753 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:52.567078 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804701a5-d841-4d0b-9a08-02c7daf2a8f6 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:52.567410 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804701a5-d841-4d0b-9a08-02c7daf2a8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:52.567690 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804701a5-d841-4d0b-9a08-02c7daf2a8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:52.587988 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-804701a5-d841-4d0b-9a08-02c7daf2a8f6 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:52.589763 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-804701a5-d841-4d0b-9a08-02c7daf2a8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:52.625675 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1054/2077] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:52 2022] POST /identity/v3/users => generated 288 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:52.627384 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43b202-9cea-47e7-822d-76be3f6fc56a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:52.630635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7f43b202-9cea-47e7-822d-76be3f6fc56a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:52.648987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43b202-9cea-47e7-822d-76be3f6fc56a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:52.711523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1024/2078] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:09:52.717544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804701a5-d841-4d0b-9a08-02c7daf2a8f6 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:52.722296 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-804701a5-d841-4d0b-9a08-02c7daf2a8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:52.803057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07837ba1-da0e-4732-8c81-9716ebe8c1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:52.809681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07837ba1-da0e-4732-8c81-9716ebe8c1e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:52.810160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07837ba1-da0e-4732-8c81-9716ebe8c1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:52.811815 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07837ba1-da0e-4732-8c81-9716ebe8c1e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:52.813040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07837ba1-da0e-4732-8c81-9716ebe8c1e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:52.817667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07837ba1-da0e-4732-8c81-9716ebe8c1e0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:52.839608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1055/2079] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:52.840457 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07837ba1-da0e-4732-8c81-9716ebe8c1e0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:52.845064 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804701a5-d841-4d0b-9a08-02c7daf2a8f6 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:52.849753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def29f20-25bf-472c-8de2-81343eacd696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:52.850790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def29f20-25bf-472c-8de2-81343eacd696 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:52.851314 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def29f20-25bf-472c-8de2-81343eacd696 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:52.851740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def29f20-25bf-472c-8de2-81343eacd696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:52.861891 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07837ba1-da0e-4732-8c81-9716ebe8c1e0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:52.864040 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def29f20-25bf-472c-8de2-81343eacd696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:52.866508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def29f20-25bf-472c-8de2-81343eacd696 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:52.867309 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2571ec1-a864-4482-a03b-365544328ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:52.868780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2571ec1-a864-4482-a03b-365544328ade admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:52.869074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2571ec1-a864-4482-a03b-365544328ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:52.869357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2571ec1-a864-4482-a03b-365544328ade admin admin] PATH_INFO: `/v3/projects/9665ba042f394c018cfb4b597da226bf/users/ce346b6a087c43009f1926240f211717/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:52.874789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:52.882800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1025/2080] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:52.890024 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2571ec1-a864-4482-a03b-365544328ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=9665ba042f394c018cfb4b597da226bf, user_id=ce346b6a087c43009f1926240f211717, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:52.893416 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2571ec1-a864-4482-a03b-365544328ade admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:52.989183 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1056/2081] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:52 2022] PUT /identity/v3/projects/9665ba042f394c018cfb4b597da226bf/users/ce346b6a087c43009f1926240f211717/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:52.993977 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def29f20-25bf-472c-8de2-81343eacd696 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:53.014085 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def29f20-25bf-472c-8de2-81343eacd696 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:53.084883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8dbebc-d017-4901-ba2c-72bca3e5c23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:53.085576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8dbebc-d017-4901-ba2c-72bca3e5c23c admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:53.085857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8dbebc-d017-4901-ba2c-72bca3e5c23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:53.086131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8dbebc-d017-4901-ba2c-72bca3e5c23c admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:53.086937 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8dbebc-d017-4901-ba2c-72bca3e5c23c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:53.088889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8dbebc-d017-4901-ba2c-72bca3e5c23c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:53.104239 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1026/2082] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:53.108277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2571ec1-a864-4482-a03b-365544328ade admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:53.121160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2571ec1-a864-4482-a03b-365544328ade admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:53.132292 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79befea-5a12-461c-b45a-b9cc00fa2a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:53.133315 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79befea-5a12-461c-b45a-b9cc00fa2a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:53.134507 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79befea-5a12-461c-b45a-b9cc00fa2a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:53.134879 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79befea-5a12-461c-b45a-b9cc00fa2a29 admin admin] PATH_INFO: `/v3/projects/9665ba042f394c018cfb4b597da226bf/users/ce346b6a087c43009f1926240f211717/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:53.138593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79befea-5a12-461c-b45a-b9cc00fa2a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9665ba042f394c018cfb4b597da226bf, user_id=ce346b6a087c43009f1926240f211717, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:53.142304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79befea-5a12-461c-b45a-b9cc00fa2a29 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:53.166847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1057/2083] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:53 2022] PUT /identity/v3/projects/9665ba042f394c018cfb4b597da226bf/users/ce346b6a087c43009f1926240f211717/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:53.169569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a927ead-5ed4-44dd-b869-ff961fcb1db8 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:53.170027 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a927ead-5ed4-44dd-b869-ff961fcb1db8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:53.173992 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a927ead-5ed4-44dd-b869-ff961fcb1db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:53.213388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-8a927ead-5ed4-44dd-b869-ff961fcb1db8 None None] MFA Rules not processed for user `ce346b6a087c43009f1926240f211717`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:53.215228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8a927ead-5ed4-44dd-b869-ff961fcb1db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:53.348893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1027/2084] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:53 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:53.355495 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79befea-5a12-461c-b45a-b9cc00fa2a29 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:53.357994 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a79befea-5a12-461c-b45a-b9cc00fa2a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:53.435659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:53.469905 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79befea-5a12-461c-b45a-b9cc00fa2a29 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:53.475895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b123b1-ca06-44bf-b172-40caee4885ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:53.476585 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b123b1-ca06-44bf-b172-40caee4885ec service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:53.476879 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b123b1-ca06-44bf-b172-40caee4885ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:53.477168 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b123b1-ca06-44bf-b172-40caee4885ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:53.491714 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b123b1-ca06-44bf-b172-40caee4885ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:53.493380 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b123b1-ca06-44bf-b172-40caee4885ec service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:53.508337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1058/2085] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:53 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:53.764686 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a927ead-5ed4-44dd-b869-ff961fcb1db8 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:53.766309 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8a927ead-5ed4-44dd-b869-ff961fcb1db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:53.774117 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b123b1-ca06-44bf-b172-40caee4885ec service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:53.785521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b123b1-ca06-44bf-b172-40caee4885ec service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:53.803449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1377781-ac05-42e9-ba0a-5b06aa063ae6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:53.804156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1377781-ac05-42e9-ba0a-5b06aa063ae6 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:53.804463 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1377781-ac05-42e9-ba0a-5b06aa063ae6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:53.804748 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1377781-ac05-42e9-ba0a-5b06aa063ae6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:53.808400 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b1377781-ac05-42e9-ba0a-5b06aa063ae6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:53.935330 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a927ead-5ed4-44dd-b869-ff961fcb1db8 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:53.946413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f7a4b-cc21-4ba8-9022-b774e97ebdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:53.947757 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f7a4b-cc21-4ba8-9022-b774e97ebdce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:53.950827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f7a4b-cc21-4ba8-9022-b774e97ebdce admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:53.950827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f7a4b-cc21-4ba8-9022-b774e97ebdce admin admin] PATH_INFO: `/v3/projects/25afb3f130ba44fb9b363fda6f262d70` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:53.952265 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f7a4b-cc21-4ba8-9022-b774e97ebdce admin admin] RBAC: Authorizing `identity:delete_project(project_id=25afb3f130ba44fb9b363fda6f262d70)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:53.955975 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f7a4b-cc21-4ba8-9022-b774e97ebdce admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:53.965100 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1377781-ac05-42e9-ba0a-5b06aa063ae6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:53.965721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1377781-ac05-42e9-ba0a-5b06aa063ae6 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:53.991243 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1059/2086] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:53 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:09:54.011687 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:54.035451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1377781-ac05-42e9-ba0a-5b06aa063ae6 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:54.037790 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b1377781-ac05-42e9-ba0a-5b06aa063ae6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:54.061428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1377781-ac05-42e9-ba0a-5b06aa063ae6 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:54.075373 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc6b0d-3f1b-4068-ae6d-044e7a2dbd75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:54.075943 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc6b0d-3f1b-4068-ae6d-044e7a2dbd75 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:54.076236 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc6b0d-3f1b-4068-ae6d-044e7a2dbd75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:54.076522 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc6b0d-3f1b-4068-ae6d-044e7a2dbd75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:54.082964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f8cc6b0d-3f1b-4068-ae6d-044e7a2dbd75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:54.128882 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d69f7a4b-cc21-4ba8-9022-b774e97ebdce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25afb3f130ba44fb9b363fda6f262d70', 'request_id': 'req-d69f7a4b-cc21-4ba8-9022-b774e97ebdce'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:54.207281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc6b0d-3f1b-4068-ae6d-044e7a2dbd75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:54.208745 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc6b0d-3f1b-4068-ae6d-044e7a2dbd75 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:54.233823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d69f7a4b-cc21-4ba8-9022-b774e97ebdce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25afb3f130ba44fb9b363fda6f262d70', 'request_id': 'req-d69f7a4b-cc21-4ba8-9022-b774e97ebdce'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:54.236760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1060/2087] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:54.237413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d69f7a4b-cc21-4ba8-9022-b774e97ebdce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25afb3f130ba44fb9b363fda6f262d70', 'request_id': 'req-d69f7a4b-cc21-4ba8-9022-b774e97ebdce'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:54.261298 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d69f7a4b-cc21-4ba8-9022-b774e97ebdce admin admin] The token cache is being invalidate because project 25afb3f130ba44fb9b363fda6f262d70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:54.261721 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d69f7a4b-cc21-4ba8-9022-b774e97ebdce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:54.262453 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d69f7a4b-cc21-4ba8-9022-b774e97ebdce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:54.264278 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1028/2088] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:53 2022] DELETE /identity/v3/projects/25afb3f130ba44fb9b363fda6f262d70 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:54.550655 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcd8ea-8580-4865-ab91-dfb27e4154ef None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:54.551694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcd8ea-8580-4865-ab91-dfb27e4154ef None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:54.552172 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcd8ea-8580-4865-ab91-dfb27e4154ef None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:54.557369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1061/2089] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 14 19:09:54 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:09:54.574462 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298a7c9-9681-44f1-b997-06a7dcface29 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:54.574851 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298a7c9-9681-44f1-b997-06a7dcface29 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:54.575719 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298a7c9-9681-44f1-b997-06a7dcface29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:54.606075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b298a7c9-9681-44f1-b997-06a7dcface29 None None] MFA Rules not processed for user `c577fa8670e34449a80de0c8e0a15a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:54.607487 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b298a7c9-9681-44f1-b997-06a7dcface29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:54.689312 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1029/2090] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 14 19:09:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:54.702522 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcd8ea-8580-4865-ab91-dfb27e4154ef None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:54.704521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cbfcd8ea-8580-4865-ab91-dfb27e4154ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:54.716930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298a7c9-9681-44f1-b997-06a7dcface29 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:54.718363 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b298a7c9-9681-44f1-b997-06a7dcface29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:54.743998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcd8ea-8580-4865-ab91-dfb27e4154ef None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:54.751004 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475b809-35ba-4e10-9b19-d1bcb25c9526 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c577fa8670e34449a80de0c8e0a15a8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:54.751855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475b809-35ba-4e10-9b19-d1bcb25c9526 service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:54.752509 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475b809-35ba-4e10-9b19-d1bcb25c9526 service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:54.752892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475b809-35ba-4e10-9b19-d1bcb25c9526 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:54.768472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8475b809-35ba-4e10-9b19-d1bcb25c9526 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:54.771425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8475b809-35ba-4e10-9b19-d1bcb25c9526 service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:54.791327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1062/2091] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:54.827666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298a7c9-9681-44f1-b997-06a7dcface29 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:54.833070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8a89c-db51-4003-b930-ecb166cae881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:54.834138 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8a89c-db51-4003-b930-ecb166cae881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:54.834578 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8a89c-db51-4003-b930-ecb166cae881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:54.834914 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8a89c-db51-4003-b930-ecb166cae881 admin admin] PATH_INFO: `/v3/users/4a5c22d1844642418e4478006c5fb1ca` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:54.837387 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8a89c-db51-4003-b930-ecb166cae881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a5c22d1844642418e4478006c5fb1ca)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:54.839577 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8a89c-db51-4003-b930-ecb166cae881 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:55.058640 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-63e8a89c-db51-4003-b930-ecb166cae881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a5c22d1844642418e4478006c5fb1ca'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:55.061002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:55.068657 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-63e8a89c-db51-4003-b930-ecb166cae881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a5c22d1844642418e4478006c5fb1ca'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:55.069455 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-63e8a89c-db51-4003-b930-ecb166cae881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a5c22d1844642418e4478006c5fb1ca'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:55.070137 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-63e8a89c-db51-4003-b930-ecb166cae881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a5c22d1844642418e4478006c5fb1ca'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:55.084332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-63e8a89c-db51-4003-b930-ecb166cae881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a5c22d1844642418e4478006c5fb1ca'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:55.173320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1030/2092] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:54 2022] DELETE /identity/v3/users/4a5c22d1844642418e4478006c5fb1ca => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:55.307638 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0e53e-779c-4b59-943b-1d5f9cfe3fc5 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:55.308216 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0e53e-779c-4b59-943b-1d5f9cfe3fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:55.308216 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0e53e-779c-4b59-943b-1d5f9cfe3fc5 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:55.309757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1063/2093] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 14 19:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:09:55.318495 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274355b-f063-4004-ad2a-2cacd82cddf0 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:55.323908 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274355b-f063-4004-ad2a-2cacd82cddf0 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:55.323908 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274355b-f063-4004-ad2a-2cacd82cddf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:55.345574 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4274355b-f063-4004-ad2a-2cacd82cddf0 None None] MFA Rules not processed for user `c577fa8670e34449a80de0c8e0a15a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:55.346948 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4274355b-f063-4004-ad2a-2cacd82cddf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:55.418535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1031/2094] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 14 19:09:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:55.429618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0e53e-779c-4b59-943b-1d5f9cfe3fc5 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:55.431364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-14c0e53e-779c-4b59-943b-1d5f9cfe3fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:55.445554 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274355b-f063-4004-ad2a-2cacd82cddf0 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:55.452524 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4274355b-f063-4004-ad2a-2cacd82cddf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:55.474508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0e53e-779c-4b59-943b-1d5f9cfe3fc5 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:55.483999 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ce435-3d99-41e2-b291-2194ea793589 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c577fa8670e34449a80de0c8e0a15a8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:55.485630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ce435-3d99-41e2-b291-2194ea793589 service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:55.487117 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ce435-3d99-41e2-b291-2194ea793589 service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:55.487498 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ce435-3d99-41e2-b291-2194ea793589 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:55.503987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ce435-3d99-41e2-b291-2194ea793589 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:55.506126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ce435-3d99-41e2-b291-2194ea793589 service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:55.528765 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1064/2095] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:55.541701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274355b-f063-4004-ad2a-2cacd82cddf0 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:55.553888 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d46836-eafb-4cf5-890a-450251b6875c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:55.555290 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d46836-eafb-4cf5-890a-450251b6875c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:55.556035 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d46836-eafb-4cf5-890a-450251b6875c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:55.556534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d46836-eafb-4cf5-890a-450251b6875c admin admin] PATH_INFO: `/v3/projects/43a3b44b0a5f40b3955ce8027cd1cc35` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:55.558149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d46836-eafb-4cf5-890a-450251b6875c admin admin] RBAC: Authorizing `identity:delete_project(project_id=43a3b44b0a5f40b3955ce8027cd1cc35)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:55.560569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d46836-eafb-4cf5-890a-450251b6875c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:55.724057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-01d46836-eafb-4cf5-890a-450251b6875c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43a3b44b0a5f40b3955ce8027cd1cc35', 'request_id': 'req-01d46836-eafb-4cf5-890a-450251b6875c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:55.865478 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-01d46836-eafb-4cf5-890a-450251b6875c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43a3b44b0a5f40b3955ce8027cd1cc35', 'request_id': 'req-01d46836-eafb-4cf5-890a-450251b6875c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:55.873978 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-01d46836-eafb-4cf5-890a-450251b6875c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43a3b44b0a5f40b3955ce8027cd1cc35', 'request_id': 'req-01d46836-eafb-4cf5-890a-450251b6875c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:55.876113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:55.904556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-01d46836-eafb-4cf5-890a-450251b6875c admin admin] The token cache is being invalidate because project 43a3b44b0a5f40b3955ce8027cd1cc35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:55.904556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-01d46836-eafb-4cf5-890a-450251b6875c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:55.904918 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-01d46836-eafb-4cf5-890a-450251b6875c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:55.906620 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1032/2096] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:55 2022] DELETE /identity/v3/projects/43a3b44b0a5f40b3955ce8027cd1cc35 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:55.914412 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe3f0e-5ae8-41ec-9fc9-bd22cf99ae72 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:55.915085 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe3f0e-5ae8-41ec-9fc9-bd22cf99ae72 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:55.915085 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe3f0e-5ae8-41ec-9fc9-bd22cf99ae72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:55.938401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9ffe3f0e-5ae8-41ec-9fc9-bd22cf99ae72 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:55.939675 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9ffe3f0e-5ae8-41ec-9fc9-bd22cf99ae72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:56.013332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:56.041630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1065/2097] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:56.050812 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59f40c-d0c8-4b42-ad22-a46319107c24 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:56.051597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59f40c-d0c8-4b42-ad22-a46319107c24 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:56.052109 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59f40c-d0c8-4b42-ad22-a46319107c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:56.082557 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-2f59f40c-d0c8-4b42-ad22-a46319107c24 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:56.083875 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2f59f40c-d0c8-4b42-ad22-a46319107c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:56.093033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1033/2098] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:56.098166 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe3f0e-5ae8-41ec-9fc9-bd22cf99ae72 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:56.109734 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe3f0e-5ae8-41ec-9fc9-bd22cf99ae72 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:56.114546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56bdcb-69f6-4545-b892-e4b18277f2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:56.115300 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56bdcb-69f6-4545-b892-e4b18277f2f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:56.118908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56bdcb-69f6-4545-b892-e4b18277f2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:56.118908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56bdcb-69f6-4545-b892-e4b18277f2f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:56.118908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56bdcb-69f6-4545-b892-e4b18277f2f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:09:56.118908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56bdcb-69f6-4545-b892-e4b18277f2f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:56.118908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56bdcb-69f6-4545-b892-e4b18277f2f0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:56.132928 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1066/2099] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:09:56.138053 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59f40c-d0c8-4b42-ad22-a46319107c24 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:56.155881 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59f40c-d0c8-4b42-ad22-a46319107c24 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:56.170401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245fe30-5f42-4028-bf81-9bf84f3a6c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:56.172141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245fe30-5f42-4028-bf81-9bf84f3a6c8e admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:56.172639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245fe30-5f42-4028-bf81-9bf84f3a6c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:56.172934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245fe30-5f42-4028-bf81-9bf84f3a6c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:56.174027 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a245fe30-5f42-4028-bf81-9bf84f3a6c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:56.176464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a245fe30-5f42-4028-bf81-9bf84f3a6c8e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:56.326362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56bdcb-69f6-4545-b892-e4b18277f2f0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:56.334428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1034/2100] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:56 2022] POST /identity/v3/projects => generated 383 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:56.366640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56bdcb-69f6-4545-b892-e4b18277f2f0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:56.480606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7c603-cf8b-49ce-8de2-6f27cd89db65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:56.481644 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7c603-cf8b-49ce-8de2-6f27cd89db65 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:56.481930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7c603-cf8b-49ce-8de2-6f27cd89db65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:56.482206 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7c603-cf8b-49ce-8de2-6f27cd89db65 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:56.482903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7c603-cf8b-49ce-8de2-6f27cd89db65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:56.485062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7c603-cf8b-49ce-8de2-6f27cd89db65 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:56.517260 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245fe30-5f42-4028-bf81-9bf84f3a6c8e admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:56.521703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a245fe30-5f42-4028-bf81-9bf84f3a6c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:56.536312 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245fe30-5f42-4028-bf81-9bf84f3a6c8e admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:56.556396 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cdc46-85e2-45fa-bf9b-ca2eea9bfa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:56.557400 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cdc46-85e2-45fa-bf9b-ca2eea9bfa63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:56.559708 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cdc46-85e2-45fa-bf9b-ca2eea9bfa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:56.560080 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cdc46-85e2-45fa-bf9b-ca2eea9bfa63 admin admin] PATH_INFO: `/v3/users/4eb6edde1b5e430e92212213de710a58` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:56.562560 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63cdc46-85e2-45fa-bf9b-ca2eea9bfa63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eb6edde1b5e430e92212213de710a58)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:56.564664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63cdc46-85e2-45fa-bf9b-ca2eea9bfa63 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:56.608570 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1067/2101] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:56 2022] POST /identity/v3/users => generated 292 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:56.613473 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7c603-cf8b-49ce-8de2-6f27cd89db65 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:56.624997 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7c603-cf8b-49ce-8de2-6f27cd89db65 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:56.636524 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b8720-1de8-458c-8c4e-9a286d40bbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:56.637243 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b8720-1de8-458c-8c4e-9a286d40bbe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:56.637532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b8720-1de8-458c-8c4e-9a286d40bbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:56.637814 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b8720-1de8-458c-8c4e-9a286d40bbe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:56.638558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b8720-1de8-458c-8c4e-9a286d40bbe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:56.640273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b8720-1de8-458c-8c4e-9a286d40bbe5 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:56.657111 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1068/2102] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:56.657111 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b8720-1de8-458c-8c4e-9a286d40bbe5 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:56.667631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b8720-1de8-458c-8c4e-9a286d40bbe5 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:56.672433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08874143-24ee-4654-929e-cc26a49adbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:56.673119 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08874143-24ee-4654-929e-cc26a49adbb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:56.673410 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08874143-24ee-4654-929e-cc26a49adbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:56.673951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08874143-24ee-4654-929e-cc26a49adbb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:56.674495 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08874143-24ee-4654-929e-cc26a49adbb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:56.676155 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08874143-24ee-4654-929e-cc26a49adbb0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:56.694167 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1069/2103] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:56.699485 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08874143-24ee-4654-929e-cc26a49adbb0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:56.709669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08874143-24ee-4654-929e-cc26a49adbb0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:56.715154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd08723-b102-467b-a211-1e83e7b21ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:56.716079 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd08723-b102-467b-a211-1e83e7b21ab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:56.716368 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd08723-b102-467b-a211-1e83e7b21ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:56.716648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd08723-b102-467b-a211-1e83e7b21ab9 admin admin] PATH_INFO: `/v3/projects/0901d3f494e6448491219e35cc999824/users/ff0c04a841c9411fb6c1e85a83896269/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:56.734608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd08723-b102-467b-a211-1e83e7b21ab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0901d3f494e6448491219e35cc999824, user_id=ff0c04a841c9411fb6c1e85a83896269, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:56.737443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd08723-b102-467b-a211-1e83e7b21ab9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:56.777821 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b63cdc46-85e2-45fa-bf9b-ca2eea9bfa63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eb6edde1b5e430e92212213de710a58'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:56.785081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b63cdc46-85e2-45fa-bf9b-ca2eea9bfa63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eb6edde1b5e430e92212213de710a58'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:56.785793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b63cdc46-85e2-45fa-bf9b-ca2eea9bfa63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eb6edde1b5e430e92212213de710a58'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:56.786388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b63cdc46-85e2-45fa-bf9b-ca2eea9bfa63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eb6edde1b5e430e92212213de710a58'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:56.799523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b63cdc46-85e2-45fa-bf9b-ca2eea9bfa63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eb6edde1b5e430e92212213de710a58'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:56.853132 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1070/2104] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:56 2022] PUT /identity/v3/projects/0901d3f494e6448491219e35cc999824/users/ff0c04a841c9411fb6c1e85a83896269/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:56.873459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae304e-c2b2-47de-b378-bcf4f5ba5a34 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:56.873814 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae304e-c2b2-47de-b378-bcf4f5ba5a34 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:56.874101 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae304e-c2b2-47de-b378-bcf4f5ba5a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:56.881520 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1035/2105] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:56 2022] DELETE /identity/v3/users/4eb6edde1b5e430e92212213de710a58 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:56.915952 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-24ae304e-c2b2-47de-b378-bcf4f5ba5a34 None None] MFA Rules not processed for user `ff0c04a841c9411fb6c1e85a83896269`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:56.915952 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-24ae304e-c2b2-47de-b378-bcf4f5ba5a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:56.998438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1071/2106] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:09:57.016662 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cdc46-85e2-45fa-bf9b-ca2eea9bfa63 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:57.018135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b63cdc46-85e2-45fa-bf9b-ca2eea9bfa63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:57.086948 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cdc46-85e2-45fa-bf9b-ca2eea9bfa63 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:57.091128 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24032260-f79e-4f56-a278-f0ee4f0af149 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:57.091605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24032260-f79e-4f56-a278-f0ee4f0af149 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:57.091894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24032260-f79e-4f56-a278-f0ee4f0af149 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:57.092177 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24032260-f79e-4f56-a278-f0ee4f0af149 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:57.101540 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24032260-f79e-4f56-a278-f0ee4f0af149 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:57.103002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24032260-f79e-4f56-a278-f0ee4f0af149 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:57.112861 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1036/2107] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:09:57.133207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae304e-c2b2-47de-b378-bcf4f5ba5a34 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:57.134631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-24ae304e-c2b2-47de-b378-bcf4f5ba5a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:57.142008 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:57.187184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24032260-f79e-4f56-a278-f0ee4f0af149 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:57.188818 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-24032260-f79e-4f56-a278-f0ee4f0af149 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:57.244847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae304e-c2b2-47de-b378-bcf4f5ba5a34 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:57.252921 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67b1b6-94fc-4358-84b1-d25670abe256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:57.253991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67b1b6-94fc-4358-84b1-d25670abe256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:57.254358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67b1b6-94fc-4358-84b1-d25670abe256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:57.254718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67b1b6-94fc-4358-84b1-d25670abe256 admin admin] PATH_INFO: `/v3/users/ff0c04a841c9411fb6c1e85a83896269` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:57.256667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67b1b6-94fc-4358-84b1-d25670abe256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff0c04a841c9411fb6c1e85a83896269)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:57.259142 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67b1b6-94fc-4358-84b1-d25670abe256 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:57.267455 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:57.302924 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24032260-f79e-4f56-a278-f0ee4f0af149 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:57.316293 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351d675-14d5-43a7-ae55-54c0c77664be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:57.317180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351d675-14d5-43a7-ae55-54c0c77664be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:57.317471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351d675-14d5-43a7-ae55-54c0c77664be admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:57.317761 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351d675-14d5-43a7-ae55-54c0c77664be admin admin] PATH_INFO: `/v3/projects/eeef42be20a24be2a6379560083e95f1` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:57.319054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3351d675-14d5-43a7-ae55-54c0c77664be admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeef42be20a24be2a6379560083e95f1)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:57.321114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3351d675-14d5-43a7-ae55-54c0c77664be admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:57.498275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3351d675-14d5-43a7-ae55-54c0c77664be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeef42be20a24be2a6379560083e95f1', 'request_id': 'req-3351d675-14d5-43a7-ae55-54c0c77664be'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:57.498971 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1a67b1b6-94fc-4358-84b1-d25670abe256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff0c04a841c9411fb6c1e85a83896269'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:57.499659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1a67b1b6-94fc-4358-84b1-d25670abe256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff0c04a841c9411fb6c1e85a83896269'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:57.514239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1a67b1b6-94fc-4358-84b1-d25670abe256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff0c04a841c9411fb6c1e85a83896269'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:57.515152 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1a67b1b6-94fc-4358-84b1-d25670abe256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff0c04a841c9411fb6c1e85a83896269'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:57.523693 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1a67b1b6-94fc-4358-84b1-d25670abe256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff0c04a841c9411fb6c1e85a83896269'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:57.607067 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3351d675-14d5-43a7-ae55-54c0c77664be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeef42be20a24be2a6379560083e95f1', 'request_id': 'req-3351d675-14d5-43a7-ae55-54c0c77664be'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:57.613788 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3351d675-14d5-43a7-ae55-54c0c77664be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeef42be20a24be2a6379560083e95f1', 'request_id': 'req-3351d675-14d5-43a7-ae55-54c0c77664be'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:57.634268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3351d675-14d5-43a7-ae55-54c0c77664be admin admin] The token cache is being invalidate because project eeef42be20a24be2a6379560083e95f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:57.634786 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3351d675-14d5-43a7-ae55-54c0c77664be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:57.635677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3351d675-14d5-43a7-ae55-54c0c77664be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:57.637860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1037/2108] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:57 2022] DELETE /identity/v3/projects/eeef42be20a24be2a6379560083e95f1 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:57.642448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351d675-14d5-43a7-ae55-54c0c77664be admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:57.644570 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3351d675-14d5-43a7-ae55-54c0c77664be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:57.696051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1072/2109] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:57 2022] DELETE /identity/v3/users/ff0c04a841c9411fb6c1e85a83896269 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:57.703635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ab56c-676a-427a-a9e0-034174ea6f06 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:57.704010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ab56c-676a-427a-a9e0-034174ea6f06 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:57.704722 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ab56c-676a-427a-a9e0-034174ea6f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:57.724743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-df3ab56c-676a-427a-a9e0-034174ea6f06 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:57.726213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-df3ab56c-676a-427a-a9e0-034174ea6f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:57.728710 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351d675-14d5-43a7-ae55-54c0c77664be admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:57.806277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1073/2110] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:57.808170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f268eb-f82b-43d1-9cfe-e61e51ba68e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:57.809039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f268eb-f82b-43d1-9cfe-e61e51ba68e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:57.809324 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f268eb-f82b-43d1-9cfe-e61e51ba68e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:57.809603 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f268eb-f82b-43d1-9cfe-e61e51ba68e6 admin admin] PATH_INFO: `/v3/users/0893b04c88d84c378522f4531e12cf9e` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:57.810964 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f268eb-f82b-43d1-9cfe-e61e51ba68e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0893b04c88d84c378522f4531e12cf9e)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:57.813151 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f268eb-f82b-43d1-9cfe-e61e51ba68e6 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:57.822756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ab56c-676a-427a-a9e0-034174ea6f06 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:57.826239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-df3ab56c-676a-427a-a9e0-034174ea6f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:57.837770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:57.859937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:57.891286 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ab56c-676a-427a-a9e0-034174ea6f06 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:57.895277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120507c2-9d1f-41ec-81bb-9212edd1263f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:57.895887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120507c2-9d1f-41ec-81bb-9212edd1263f service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:57.896177 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120507c2-9d1f-41ec-81bb-9212edd1263f service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:57.896459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120507c2-9d1f-41ec-81bb-9212edd1263f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:57.906672 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120507c2-9d1f-41ec-81bb-9212edd1263f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:57.908306 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120507c2-9d1f-41ec-81bb-9212edd1263f service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:57.918037 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1074/2111] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:09:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:09:58.061221 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c9f268eb-f82b-43d1-9cfe-e61e51ba68e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0893b04c88d84c378522f4531e12cf9e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:58.068360 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c9f268eb-f82b-43d1-9cfe-e61e51ba68e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0893b04c88d84c378522f4531e12cf9e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:58.069056 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c9f268eb-f82b-43d1-9cfe-e61e51ba68e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0893b04c88d84c378522f4531e12cf9e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:58.069677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c9f268eb-f82b-43d1-9cfe-e61e51ba68e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0893b04c88d84c378522f4531e12cf9e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:58.083441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c9f268eb-f82b-43d1-9cfe-e61e51ba68e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0893b04c88d84c378522f4531e12cf9e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:58.179092 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1038/2112] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:09:57 2022] DELETE /identity/v3/users/0893b04c88d84c378522f4531e12cf9e => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:58.728751 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120507c2-9d1f-41ec-81bb-9212edd1263f service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:58.745771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f268eb-f82b-43d1-9cfe-e61e51ba68e6 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:58.747623 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120507c2-9d1f-41ec-81bb-9212edd1263f service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:58.749162 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c9f268eb-f82b-43d1-9cfe-e61e51ba68e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:58.920316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9750a-b36c-4c76-a1b7-ef1c5bcd1bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:58.921516 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f268eb-f82b-43d1-9cfe-e61e51ba68e6 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:58.924717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9750a-b36c-4c76-a1b7-ef1c5bcd1bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:58.925055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9750a-b36c-4c76-a1b7-ef1c5bcd1bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:58.926781 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9750a-b36c-4c76-a1b7-ef1c5bcd1bdf admin admin] PATH_INFO: `/v3/projects/09814c09a8594e62b20c5581c1eae8df` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:58.927153 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b1320-3635-41d8-953d-56791bc12106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:58.929377 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b1320-3635-41d8-953d-56791bc12106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:58.929670 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b1320-3635-41d8-953d-56791bc12106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:58.930012 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b1320-3635-41d8-953d-56791bc12106 admin admin] PATH_INFO: `/v3/projects/0901d3f494e6448491219e35cc999824` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:58.930948 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9750a-b36c-4c76-a1b7-ef1c5bcd1bdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=09814c09a8594e62b20c5581c1eae8df)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:58.931764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b1320-3635-41d8-953d-56791bc12106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0901d3f494e6448491219e35cc999824)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:58.934586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b1320-3635-41d8-953d-56791bc12106 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:58.940076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9750a-b36c-4c76-a1b7-ef1c5bcd1bdf admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:59.083133 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-088b1320-3635-41d8-953d-56791bc12106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0901d3f494e6448491219e35cc999824', 'request_id': 'req-088b1320-3635-41d8-953d-56791bc12106'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:59.087969 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-80b9750a-b36c-4c76-a1b7-ef1c5bcd1bdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09814c09a8594e62b20c5581c1eae8df', 'request_id': 'req-80b9750a-b36c-4c76-a1b7-ef1c5bcd1bdf'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:59.184702 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-088b1320-3635-41d8-953d-56791bc12106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0901d3f494e6448491219e35cc999824', 'request_id': 'req-088b1320-3635-41d8-953d-56791bc12106'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:59.190980 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-088b1320-3635-41d8-953d-56791bc12106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0901d3f494e6448491219e35cc999824', 'request_id': 'req-088b1320-3635-41d8-953d-56791bc12106'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:59.203933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-088b1320-3635-41d8-953d-56791bc12106 admin admin] The token cache is being invalidate because project 0901d3f494e6448491219e35cc999824 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:59.204232 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-088b1320-3635-41d8-953d-56791bc12106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:59.204940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-088b1320-3635-41d8-953d-56791bc12106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:59.206548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1039/2113] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:58 2022] DELETE /identity/v3/projects/0901d3f494e6448491219e35cc999824 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:09:59.212724 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e702b-57ff-4e9f-9d21-579efb0dfc0b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:59.213329 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e702b-57ff-4e9f-9d21-579efb0dfc0b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:59.213678 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e702b-57ff-4e9f-9d21-579efb0dfc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:59.240255 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0c9e702b-57ff-4e9f-9d21-579efb0dfc0b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:59.241542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0c9e702b-57ff-4e9f-9d21-579efb0dfc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:59.264443 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:59.291047 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-80b9750a-b36c-4c76-a1b7-ef1c5bcd1bdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09814c09a8594e62b20c5581c1eae8df', 'request_id': 'req-80b9750a-b36c-4c76-a1b7-ef1c5bcd1bdf'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:59.298337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-80b9750a-b36c-4c76-a1b7-ef1c5bcd1bdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09814c09a8594e62b20c5581c1eae8df', 'request_id': 'req-80b9750a-b36c-4c76-a1b7-ef1c5bcd1bdf'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:59.318990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-80b9750a-b36c-4c76-a1b7-ef1c5bcd1bdf admin admin] The token cache is being invalidate because project 09814c09a8594e62b20c5581c1eae8df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:09:59.319146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1040/2114] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:59.319694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-80b9750a-b36c-4c76-a1b7-ef1c5bcd1bdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:59.324444 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-80b9750a-b36c-4c76-a1b7-ef1c5bcd1bdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:09:59.328787 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c3e3c-5b38-41ee-adf2-a31c31d9bd6a None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:59.328787 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c3e3c-5b38-41ee-adf2-a31c31d9bd6a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:59.329059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1075/2115] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:09:58 2022] DELETE /identity/v3/projects/09814c09a8594e62b20c5581c1eae8df => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:09:59.329158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c3e3c-5b38-41ee-adf2-a31c31d9bd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:59.336302 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9058edd-16a9-4753-aede-4294274bef57 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:59.336481 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9058edd-16a9-4753-aede-4294274bef57 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:59.336941 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9058edd-16a9-4753-aede-4294274bef57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:59.354606 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-006c3e3c-5b38-41ee-adf2-a31c31d9bd6a None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:59.355932 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-006c3e3c-5b38-41ee-adf2-a31c31d9bd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:59.360396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c9058edd-16a9-4753-aede-4294274bef57 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:59.362026 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c9058edd-16a9-4753-aede-4294274bef57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:59.364952 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1041/2116] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:09:59.371778 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c3e3c-5b38-41ee-adf2-a31c31d9bd6a None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:59.383210 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1076/2117] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:59.384312 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c3e3c-5b38-41ee-adf2-a31c31d9bd6a None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:59.391599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ae6c4-d566-4e36-b02c-75cab6ebb94e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:59.392166 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ae6c4-d566-4e36-b02c-75cab6ebb94e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:59.392459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ae6c4-d566-4e36-b02c-75cab6ebb94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:59.401739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c24f98-b591-441b-8cbf-9df11968a31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:59.401739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c24f98-b591-441b-8cbf-9df11968a31d admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:59.401739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c24f98-b591-441b-8cbf-9df11968a31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:59.401739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c24f98-b591-441b-8cbf-9df11968a31d admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:59.403364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c24f98-b591-441b-8cbf-9df11968a31d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:09:59.403931 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c24f98-b591-441b-8cbf-9df11968a31d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:59.405996 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c24f98-b591-441b-8cbf-9df11968a31d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:59.420399 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-f36ae6c4-d566-4e36-b02c-75cab6ebb94e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:09:59.422207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f36ae6c4-d566-4e36-b02c-75cab6ebb94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:09:59.422510 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1042/2118] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 19:09:59.426654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c24f98-b591-441b-8cbf-9df11968a31d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:59.433287 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1077/2119] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:09:59.436963 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ae6c4-d566-4e36-b02c-75cab6ebb94e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:59.442560 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c24f98-b591-441b-8cbf-9df11968a31d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:59.450415 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fa255-6514-4ceb-b0ab-83cc4c91d693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:59.451390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fa255-6514-4ceb-b0ab-83cc4c91d693 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:59.451851 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fa255-6514-4ceb-b0ab-83cc4c91d693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:59.452571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fa255-6514-4ceb-b0ab-83cc4c91d693 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:59.453411 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9fa255-6514-4ceb-b0ab-83cc4c91d693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:59.455367 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9fa255-6514-4ceb-b0ab-83cc4c91d693 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:59.456590 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ae6c4-d566-4e36-b02c-75cab6ebb94e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:59.470051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04549e5-b6c1-457b-a405-e7bcd156e091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:59.471230 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04549e5-b6c1-457b-a405-e7bcd156e091 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:59.471749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04549e5-b6c1-457b-a405-e7bcd156e091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:59.472256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04549e5-b6c1-457b-a405-e7bcd156e091 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:59.473285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04549e5-b6c1-457b-a405-e7bcd156e091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:09:59.473661 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04549e5-b6c1-457b-a405-e7bcd156e091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:59.476500 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04549e5-b6c1-457b-a405-e7bcd156e091 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:59.494001 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1078/2120] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:09:59.500764 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04549e5-b6c1-457b-a405-e7bcd156e091 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:59.508765 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04549e5-b6c1-457b-a405-e7bcd156e091 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:59.513558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b472b7b-3493-4ee3-8200-d7b100b8150e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:59.514222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b472b7b-3493-4ee3-8200-d7b100b8150e admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:59.514512 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b472b7b-3493-4ee3-8200-d7b100b8150e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:59.514791 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b472b7b-3493-4ee3-8200-d7b100b8150e admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:59.515525 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b472b7b-3493-4ee3-8200-d7b100b8150e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:59.517461 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b472b7b-3493-4ee3-8200-d7b100b8150e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:59.557421 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1043/2121] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:59 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:59.561431 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fa255-6514-4ceb-b0ab-83cc4c91d693 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:59.616961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1079/2122] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:09:59 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:09:59.622229 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b472b7b-3493-4ee3-8200-d7b100b8150e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:59.638595 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:09:59.638845 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fa255-6514-4ceb-b0ab-83cc4c91d693 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:59.722340 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b472b7b-3493-4ee3-8200-d7b100b8150e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:59.724178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c28b0-c40b-4e9b-899c-1d28c6c01d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:59.725049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c28b0-c40b-4e9b-899c-1d28c6c01d13 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:59.725491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c28b0-c40b-4e9b-899c-1d28c6c01d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:59.725787 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c28b0-c40b-4e9b-899c-1d28c6c01d13 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:59.726486 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c28b0-c40b-4e9b-899c-1d28c6c01d13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:59.728518 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c28b0-c40b-4e9b-899c-1d28c6c01d13 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:59.728744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4743f-e7d9-4bf5-8d8d-2ca1ac185fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:59.729430 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4743f-e7d9-4bf5-8d8d-2ca1ac185fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:59.729969 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4743f-e7d9-4bf5-8d8d-2ca1ac185fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:59.729969 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4743f-e7d9-4bf5-8d8d-2ca1ac185fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:59.730993 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4743f-e7d9-4bf5-8d8d-2ca1ac185fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:59.732996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4743f-e7d9-4bf5-8d8d-2ca1ac185fc1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:59.879086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1080/2123] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:59 2022] POST /identity/v3/users => generated 289 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:09:59.879322 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1044/2124] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:09:59 2022] POST /identity/v3/users => generated 292 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:09:59.883795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c28b0-c40b-4e9b-899c-1d28c6c01d13 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:59.884222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4743f-e7d9-4bf5-8d8d-2ca1ac185fc1 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:59.895074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c28b0-c40b-4e9b-899c-1d28c6c01d13 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:59.896645 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4743f-e7d9-4bf5-8d8d-2ca1ac185fc1 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:59.900763 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b182742-b96d-4c72-b422-c7b6d7fdaa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:59.901466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b182742-b96d-4c72-b422-c7b6d7fdaa43 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:59.901752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b182742-b96d-4c72-b422-c7b6d7fdaa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:59.902029 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b182742-b96d-4c72-b422-c7b6d7fdaa43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:59.902755 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b182742-b96d-4c72-b422-c7b6d7fdaa43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:59.903049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dda5ad-8708-4922-b047-b4dddab1d8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:59.903783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dda5ad-8708-4922-b047-b4dddab1d8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:59.904161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dda5ad-8708-4922-b047-b4dddab1d8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:59.904363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dda5ad-8708-4922-b047-b4dddab1d8f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:59.904476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b182742-b96d-4c72-b422-c7b6d7fdaa43 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:59.905106 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dda5ad-8708-4922-b047-b4dddab1d8f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:59.906717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dda5ad-8708-4922-b047-b4dddab1d8f4 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:59.914788 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1045/2125] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:59.919742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1081/2126] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:59.923405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b182742-b96d-4c72-b422-c7b6d7fdaa43 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:59.924044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dda5ad-8708-4922-b047-b4dddab1d8f4 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:59.935985 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b182742-b96d-4c72-b422-c7b6d7fdaa43 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:59.938354 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dda5ad-8708-4922-b047-b4dddab1d8f4 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:59.941475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42be9f7-eabd-4c9c-878b-68af58c8c750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:59.942185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42be9f7-eabd-4c9c-878b-68af58c8c750 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:59.942484 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42be9f7-eabd-4c9c-878b-68af58c8c750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:59.942983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42be9f7-eabd-4c9c-878b-68af58c8c750 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:59.943754 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42be9f7-eabd-4c9c-878b-68af58c8c750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:59.946102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efec82a-3082-4cc8-9ce8-d0fd8b56d62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:59.946102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efec82a-3082-4cc8-9ce8-d0fd8b56d62d admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:59.946300 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42be9f7-eabd-4c9c-878b-68af58c8c750 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:59.946382 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efec82a-3082-4cc8-9ce8-d0fd8b56d62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:59.946382 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efec82a-3082-4cc8-9ce8-d0fd8b56d62d admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:59.946772 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efec82a-3082-4cc8-9ce8-d0fd8b56d62d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:09:59.948627 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efec82a-3082-4cc8-9ce8-d0fd8b56d62d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:09:59.957579 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1046/2127] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:59.963470 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1082/2128] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:09:59.966715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42be9f7-eabd-4c9c-878b-68af58c8c750 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:59.968199 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efec82a-3082-4cc8-9ce8-d0fd8b56d62d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:09:59.977741 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42be9f7-eabd-4c9c-878b-68af58c8c750 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:59.980814 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efec82a-3082-4cc8-9ce8-d0fd8b56d62d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:09:59.983142 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a961b9a-c083-46f8-a4b8-235ce6127d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:59.984124 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a961b9a-c083-46f8-a4b8-235ce6127d63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:59.984403 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a961b9a-c083-46f8-a4b8-235ce6127d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:59.984686 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a961b9a-c083-46f8-a4b8-235ce6127d63 admin admin] PATH_INFO: `/v3/projects/4cfdae029d3a4d139195ff33254b5bad/users/987ef34207774fb183b9d6a1da86893a/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:09:59.985618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9db59-39d6-418d-bd85-79df821fd4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:09:59.986535 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9db59-39d6-418d-bd85-79df821fd4de admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:09:59.986823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9db59-39d6-418d-bd85-79df821fd4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:09:59.987106 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9db59-39d6-418d-bd85-79df821fd4de admin admin] PATH_INFO: `/v3/projects/367d424ad7094fc7906f513627a95334/users/585f311a9cfa46bebc7baa787a111d83/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:00.007992 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9db59-39d6-418d-bd85-79df821fd4de admin admin] RBAC: Authorizing `identity:create_grant(project_id=367d424ad7094fc7906f513627a95334, user_id=585f311a9cfa46bebc7baa787a111d83, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:00.011612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9db59-39d6-418d-bd85-79df821fd4de admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:00.013960 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a961b9a-c083-46f8-a4b8-235ce6127d63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cfdae029d3a4d139195ff33254b5bad, user_id=987ef34207774fb183b9d6a1da86893a, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:00.017262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a961b9a-c083-46f8-a4b8-235ce6127d63 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:00.117428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1083/2129] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:59 2022] PUT /identity/v3/projects/367d424ad7094fc7906f513627a95334/users/585f311a9cfa46bebc7baa787a111d83/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:00.130126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80cffe-b412-4168-8689-a352c1d5aaa4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:00.130328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80cffe-b412-4168-8689-a352c1d5aaa4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:00.130620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80cffe-b412-4168-8689-a352c1d5aaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:00.146668 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1047/2130] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:09:59 2022] PUT /identity/v3/projects/4cfdae029d3a4d139195ff33254b5bad/users/987ef34207774fb183b9d6a1da86893a/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:00.165207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230cf1c-6e62-41cf-ba75-99c5d2649684 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:00.165518 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230cf1c-6e62-41cf-ba75-99c5d2649684 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:00.165807 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230cf1c-6e62-41cf-ba75-99c5d2649684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:00.171580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ab80cffe-b412-4168-8689-a352c1d5aaa4 None None] MFA Rules not processed for user `585f311a9cfa46bebc7baa787a111d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:00.172964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ab80cffe-b412-4168-8689-a352c1d5aaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:00.195025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f230cf1c-6e62-41cf-ba75-99c5d2649684 None None] MFA Rules not processed for user `987ef34207774fb183b9d6a1da86893a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:00.196414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f230cf1c-6e62-41cf-ba75-99c5d2649684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:00.269699 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1084/2131] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:00.281030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80cffe-b412-4168-8689-a352c1d5aaa4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:00.282451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ab80cffe-b412-4168-8689-a352c1d5aaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:00.286661 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1048/2132] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:00.303510 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230cf1c-6e62-41cf-ba75-99c5d2649684 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:00.305033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f230cf1c-6e62-41cf-ba75-99c5d2649684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:00.349203 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80cffe-b412-4168-8689-a352c1d5aaa4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:00.353162 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e5a33-56db-4e4c-87fd-1a4a300c383e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:00.353908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e5a33-56db-4e4c-87fd-1a4a300c383e service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:00.354132 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e5a33-56db-4e4c-87fd-1a4a300c383e service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:00.355040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e5a33-56db-4e4c-87fd-1a4a300c383e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:00.366063 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e5a33-56db-4e4c-87fd-1a4a300c383e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:00.368464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e5a33-56db-4e4c-87fd-1a4a300c383e service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:00.381790 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1085/2133] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:10:00.398482 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:00.409411 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230cf1c-6e62-41cf-ba75-99c5d2649684 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:00.417478 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb1221-fda0-4b66-b4c4-04b86c46da18 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5d76eae62b54521a95f36284118fada', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:00.418587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb1221-fda0-4b66-b4c4-04b86c46da18 service glance] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:00.419078 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb1221-fda0-4b66-b4c4-04b86c46da18 service glance] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:00.419907 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb1221-fda0-4b66-b4c4-04b86c46da18 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:00.437475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb1221-fda0-4b66-b4c4-04b86c46da18 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:00.439130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb1221-fda0-4b66-b4c4-04b86c46da18 service glance] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:00.452234 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1049/2134] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:10:00.468790 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e5a33-56db-4e4c-87fd-1a4a300c383e service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:00.470960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8a7e5a33-56db-4e4c-87fd-1a4a300c383e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:00.539055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:00.569713 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e5a33-56db-4e4c-87fd-1a4a300c383e service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:00.578230 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7f455-7a14-4b48-a414-505cc01f72d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:00.579380 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7f455-7a14-4b48-a414-505cc01f72d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:00.579759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7f455-7a14-4b48-a414-505cc01f72d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:00.582757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7f455-7a14-4b48-a414-505cc01f72d6 admin admin] PATH_INFO: `/v3/users/f67414953d2b432987e74a6d00d824f4` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:00.584434 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7f455-7a14-4b48-a414-505cc01f72d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f67414953d2b432987e74a6d00d824f4)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:00.586935 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7f455-7a14-4b48-a414-505cc01f72d6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:00.884348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c7a7f455-7a14-4b48-a414-505cc01f72d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f67414953d2b432987e74a6d00d824f4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:00.885065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c7a7f455-7a14-4b48-a414-505cc01f72d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f67414953d2b432987e74a6d00d824f4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:00.900302 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c7a7f455-7a14-4b48-a414-505cc01f72d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f67414953d2b432987e74a6d00d824f4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:00.901183 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c7a7f455-7a14-4b48-a414-505cc01f72d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f67414953d2b432987e74a6d00d824f4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:00.909471 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c7a7f455-7a14-4b48-a414-505cc01f72d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f67414953d2b432987e74a6d00d824f4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:01.041432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1086/2135] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:10:00 2022] DELETE /identity/v3/users/f67414953d2b432987e74a6d00d824f4 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:01.046652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d089bb-f6e0-4ab0-a056-68bb8dd7bffb None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:01.047152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d089bb-f6e0-4ab0-a056-68bb8dd7bffb None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:01.047347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d089bb-f6e0-4ab0-a056-68bb8dd7bffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:01.067729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-66d089bb-f6e0-4ab0-a056-68bb8dd7bffb None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:01.075709 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-66d089bb-f6e0-4ab0-a056-68bb8dd7bffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:01.135059 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:01.154399 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1050/2136] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:01.165852 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7f455-7a14-4b48-a414-505cc01f72d6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:01.167958 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c7a7f455-7a14-4b48-a414-505cc01f72d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:01.271657 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7f455-7a14-4b48-a414-505cc01f72d6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:01.277482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b724b02-5978-4580-8ccd-7bb381478f52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:01.278244 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b724b02-5978-4580-8ccd-7bb381478f52 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:01.278673 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b724b02-5978-4580-8ccd-7bb381478f52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:01.279121 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b724b02-5978-4580-8ccd-7bb381478f52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:01.290886 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b724b02-5978-4580-8ccd-7bb381478f52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:01.293109 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b724b02-5978-4580-8ccd-7bb381478f52 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:01.304308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1087/2137] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:10:02.014597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d089bb-f6e0-4ab0-a056-68bb8dd7bffb None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:02.016254 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-66d089bb-f6e0-4ab0-a056-68bb8dd7bffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:02.031423 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d089bb-f6e0-4ab0-a056-68bb8dd7bffb None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:02.036995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150bc0b-1a77-4b48-b103-1809016a0d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:02.040767 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150bc0b-1a77-4b48-b103-1809016a0d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:02.040767 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150bc0b-1a77-4b48-b103-1809016a0d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:02.040767 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150bc0b-1a77-4b48-b103-1809016a0d51 admin admin] PATH_INFO: `/v3/projects/0bf5ec66c66f4d9e9ed5aafc2d799350` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:02.040767 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150bc0b-1a77-4b48-b103-1809016a0d51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bf5ec66c66f4d9e9ed5aafc2d799350)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:02.041639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150bc0b-1a77-4b48-b103-1809016a0d51 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:02.151811 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f150bc0b-1a77-4b48-b103-1809016a0d51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bf5ec66c66f4d9e9ed5aafc2d799350', 'request_id': 'req-f150bc0b-1a77-4b48-b103-1809016a0d51'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:02.235497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f150bc0b-1a77-4b48-b103-1809016a0d51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bf5ec66c66f4d9e9ed5aafc2d799350', 'request_id': 'req-f150bc0b-1a77-4b48-b103-1809016a0d51'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:02.242041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f150bc0b-1a77-4b48-b103-1809016a0d51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bf5ec66c66f4d9e9ed5aafc2d799350', 'request_id': 'req-f150bc0b-1a77-4b48-b103-1809016a0d51'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:02.253743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f150bc0b-1a77-4b48-b103-1809016a0d51 admin admin] The token cache is being invalidate because project 0bf5ec66c66f4d9e9ed5aafc2d799350 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:10:02.254032 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f150bc0b-1a77-4b48-b103-1809016a0d51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:02.254700 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f150bc0b-1a77-4b48-b103-1809016a0d51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:02.256363 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1051/2138] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:10:02 2022] DELETE /identity/v3/projects/0bf5ec66c66f4d9e9ed5aafc2d799350 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:02.260515 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b724b02-5978-4580-8ccd-7bb381478f52 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:02.270709 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b724b02-5978-4580-8ccd-7bb381478f52 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:02.275051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:02.340448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf60e49-2e0c-479f-b4de-3f542f7e1c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:02.341347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf60e49-2e0c-479f-b4de-3f542f7e1c46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:02.341759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf60e49-2e0c-479f-b4de-3f542f7e1c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:02.342002 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf60e49-2e0c-479f-b4de-3f542f7e1c46 admin admin] PATH_INFO: `/v3/users/1f0dad75217e46589df50e1d7a644d75` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:02.343299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf60e49-2e0c-479f-b4de-3f542f7e1c46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f0dad75217e46589df50e1d7a644d75)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:02.345499 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf60e49-2e0c-479f-b4de-3f542f7e1c46 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:02.570339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-eaf60e49-2e0c-479f-b4de-3f542f7e1c46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f0dad75217e46589df50e1d7a644d75'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:02.571001 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-eaf60e49-2e0c-479f-b4de-3f542f7e1c46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f0dad75217e46589df50e1d7a644d75'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:02.585571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-eaf60e49-2e0c-479f-b4de-3f542f7e1c46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f0dad75217e46589df50e1d7a644d75'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:02.586295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-eaf60e49-2e0c-479f-b4de-3f542f7e1c46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f0dad75217e46589df50e1d7a644d75'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:02.595668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-eaf60e49-2e0c-479f-b4de-3f542f7e1c46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f0dad75217e46589df50e1d7a644d75'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:02.676064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1088/2139] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:10:02 2022] DELETE /identity/v3/users/1f0dad75217e46589df50e1d7a644d75 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:03.123366 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150bc0b-1a77-4b48-b103-1809016a0d51 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:03.124820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f150bc0b-1a77-4b48-b103-1809016a0d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:03.139443 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150bc0b-1a77-4b48-b103-1809016a0d51 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:03.202252 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525f720-707c-4b86-8cbf-5db2c1ceebd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:03.203196 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525f720-707c-4b86-8cbf-5db2c1ceebd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:03.203610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525f720-707c-4b86-8cbf-5db2c1ceebd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:03.203892 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525f720-707c-4b86-8cbf-5db2c1ceebd2 admin admin] PATH_INFO: `/v3/projects/9fd7fdefaeed4c8886a69652ac419c61` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:03.205251 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9525f720-707c-4b86-8cbf-5db2c1ceebd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fd7fdefaeed4c8886a69652ac419c61)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:03.207235 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9525f720-707c-4b86-8cbf-5db2c1ceebd2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:03.335329 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9525f720-707c-4b86-8cbf-5db2c1ceebd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fd7fdefaeed4c8886a69652ac419c61', 'request_id': 'req-9525f720-707c-4b86-8cbf-5db2c1ceebd2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:03.418681 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9525f720-707c-4b86-8cbf-5db2c1ceebd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fd7fdefaeed4c8886a69652ac419c61', 'request_id': 'req-9525f720-707c-4b86-8cbf-5db2c1ceebd2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:03.425065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9525f720-707c-4b86-8cbf-5db2c1ceebd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fd7fdefaeed4c8886a69652ac419c61', 'request_id': 'req-9525f720-707c-4b86-8cbf-5db2c1ceebd2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:03.437149 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9525f720-707c-4b86-8cbf-5db2c1ceebd2 admin admin] The token cache is being invalidate because project 9fd7fdefaeed4c8886a69652ac419c61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:10:03.437442 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9525f720-707c-4b86-8cbf-5db2c1ceebd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:03.438098 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9525f720-707c-4b86-8cbf-5db2c1ceebd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:03.439725 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1052/2140] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:10:03 2022] DELETE /identity/v3/projects/9fd7fdefaeed4c8886a69652ac419c61 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:03.445362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf60e49-2e0c-479f-b4de-3f542f7e1c46 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:03.453294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-eaf60e49-2e0c-479f-b4de-3f542f7e1c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:03.462915 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf60e49-2e0c-479f-b4de-3f542f7e1c46 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:03.538150 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc7fa9-4edb-4432-979e-e8bd63ecdbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:03.544866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc7fa9-4edb-4432-979e-e8bd63ecdbc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:03.544866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc7fa9-4edb-4432-979e-e8bd63ecdbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:03.544866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc7fa9-4edb-4432-979e-e8bd63ecdbc3 admin admin] PATH_INFO: `/v3/users/4fd3c1b274534d9395d8d72a75591aa3` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:03.544866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc7fa9-4edb-4432-979e-e8bd63ecdbc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fd3c1b274534d9395d8d72a75591aa3)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:03.544866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc7fa9-4edb-4432-979e-e8bd63ecdbc3 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:03.626200 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:03.706258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9dfc7fa9-4edb-4432-979e-e8bd63ecdbc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fd3c1b274534d9395d8d72a75591aa3'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:03.706916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9dfc7fa9-4edb-4432-979e-e8bd63ecdbc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fd3c1b274534d9395d8d72a75591aa3'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:03.719841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9dfc7fa9-4edb-4432-979e-e8bd63ecdbc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fd3c1b274534d9395d8d72a75591aa3'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:03.720604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9dfc7fa9-4edb-4432-979e-e8bd63ecdbc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fd3c1b274534d9395d8d72a75591aa3'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:03.727676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9dfc7fa9-4edb-4432-979e-e8bd63ecdbc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fd3c1b274534d9395d8d72a75591aa3'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:03.830965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1089/2141] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:10:03 2022] DELETE /identity/v3/users/4fd3c1b274534d9395d8d72a75591aa3 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:04.451715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525f720-707c-4b86-8cbf-5db2c1ceebd2 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:04.453769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9525f720-707c-4b86-8cbf-5db2c1ceebd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:04.463084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:04.469980 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525f720-707c-4b86-8cbf-5db2c1ceebd2 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:04.552533 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7813b-d420-452e-a4ab-f8ecf08d0415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:04.553573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7813b-d420-452e-a4ab-f8ecf08d0415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:04.553866 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7813b-d420-452e-a4ab-f8ecf08d0415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:04.554156 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7813b-d420-452e-a4ab-f8ecf08d0415 admin admin] PATH_INFO: `/v3/projects/3bfda1a3a0ae40c1a1cbe82570740227` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:04.555461 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7813b-d420-452e-a4ab-f8ecf08d0415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bfda1a3a0ae40c1a1cbe82570740227)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:04.557511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7813b-d420-452e-a4ab-f8ecf08d0415 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:04.729896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-64f7813b-d420-452e-a4ab-f8ecf08d0415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bfda1a3a0ae40c1a1cbe82570740227', 'request_id': 'req-64f7813b-d420-452e-a4ab-f8ecf08d0415'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:04.806991 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-64f7813b-d420-452e-a4ab-f8ecf08d0415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bfda1a3a0ae40c1a1cbe82570740227', 'request_id': 'req-64f7813b-d420-452e-a4ab-f8ecf08d0415'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:04.815381 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-64f7813b-d420-452e-a4ab-f8ecf08d0415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bfda1a3a0ae40c1a1cbe82570740227', 'request_id': 'req-64f7813b-d420-452e-a4ab-f8ecf08d0415'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:04.829625 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-64f7813b-d420-452e-a4ab-f8ecf08d0415 admin admin] The token cache is being invalidate because project 3bfda1a3a0ae40c1a1cbe82570740227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:10:04.830139 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-64f7813b-d420-452e-a4ab-f8ecf08d0415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:04.830930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-64f7813b-d420-452e-a4ab-f8ecf08d0415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:04.832887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1053/2142] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:10:04 2022] DELETE /identity/v3/projects/3bfda1a3a0ae40c1a1cbe82570740227 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:04.838005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837114e-8133-4136-b08b-f0bfca5c013e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:04.842609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837114e-8133-4136-b08b-f0bfca5c013e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:04.842609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837114e-8133-4136-b08b-f0bfca5c013e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:04.861592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5837114e-8133-4136-b08b-f0bfca5c013e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:04.863112 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5837114e-8133-4136-b08b-f0bfca5c013e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:04.938420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1090/2143] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:04.948455 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048aa55d-0e80-469e-a086-5768cf653390 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:04.948907 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048aa55d-0e80-469e-a086-5768cf653390 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:04.949289 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048aa55d-0e80-469e-a086-5768cf653390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:04.968938 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-048aa55d-0e80-469e-a086-5768cf653390 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:04.970332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-048aa55d-0e80-469e-a086-5768cf653390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:04.979736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1054/2144] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:04.984488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837114e-8133-4136-b08b-f0bfca5c013e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:04.996046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837114e-8133-4136-b08b-f0bfca5c013e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:05.001901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271481a-6c3f-4efd-a8d8-d3f650b34cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:05.002579 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271481a-6c3f-4efd-a8d8-d3f650b34cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:05.002868 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271481a-6c3f-4efd-a8d8-d3f650b34cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:05.003146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271481a-6c3f-4efd-a8d8-d3f650b34cd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:05.003922 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271481a-6c3f-4efd-a8d8-d3f650b34cd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:10:05.004232 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271481a-6c3f-4efd-a8d8-d3f650b34cd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:05.005902 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271481a-6c3f-4efd-a8d8-d3f650b34cd1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:05.021002 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1091/2145] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:10:05.025795 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048aa55d-0e80-469e-a086-5768cf653390 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:05.035811 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048aa55d-0e80-469e-a086-5768cf653390 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:05.040635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581e75a-5fd6-40e5-9862-00b186f25473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:05.041366 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581e75a-5fd6-40e5-9862-00b186f25473 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:05.042253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581e75a-5fd6-40e5-9862-00b186f25473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:05.042543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581e75a-5fd6-40e5-9862-00b186f25473 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:05.043279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581e75a-5fd6-40e5-9862-00b186f25473 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:05.045215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581e75a-5fd6-40e5-9862-00b186f25473 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:05.124839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1055/2146] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:10:05 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:10:05.128910 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271481a-6c3f-4efd-a8d8-d3f650b34cd1 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:05.139306 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271481a-6c3f-4efd-a8d8-d3f650b34cd1 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:05.204602 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7913f-9fad-47b3-a4e6-9a522f1b0c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:05.205417 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7913f-9fad-47b3-a4e6-9a522f1b0c65 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:05.205594 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7913f-9fad-47b3-a4e6-9a522f1b0c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:05.205850 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7913f-9fad-47b3-a4e6-9a522f1b0c65 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:05.206586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7913f-9fad-47b3-a4e6-9a522f1b0c65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:05.208635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7913f-9fad-47b3-a4e6-9a522f1b0c65 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:05.268111 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1092/2147] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:10:05 2022] POST /identity/v3/users => generated 292 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:10:05.274274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581e75a-5fd6-40e5-9862-00b186f25473 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:05.285876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581e75a-5fd6-40e5-9862-00b186f25473 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:05.297020 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21342888-56e6-4a73-aaf3-cb3dbac94ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:05.297721 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21342888-56e6-4a73-aaf3-cb3dbac94ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:05.298039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21342888-56e6-4a73-aaf3-cb3dbac94ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:05.298317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21342888-56e6-4a73-aaf3-cb3dbac94ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:05.299057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21342888-56e6-4a73-aaf3-cb3dbac94ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:05.300787 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21342888-56e6-4a73-aaf3-cb3dbac94ba3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:05.314952 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1056/2148] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:10:05.320173 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7913f-9fad-47b3-a4e6-9a522f1b0c65 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:05.335546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7913f-9fad-47b3-a4e6-9a522f1b0c65 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:05.341056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba76b7-4b9c-4f27-82d9-124cd4dfeebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:05.341756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba76b7-4b9c-4f27-82d9-124cd4dfeebb admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:05.342136 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba76b7-4b9c-4f27-82d9-124cd4dfeebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:05.342453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba76b7-4b9c-4f27-82d9-124cd4dfeebb admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:05.343463 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba76b7-4b9c-4f27-82d9-124cd4dfeebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:05.345433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba76b7-4b9c-4f27-82d9-124cd4dfeebb admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:05.358579 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1093/2149] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:10:05.363124 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21342888-56e6-4a73-aaf3-cb3dbac94ba3 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:05.373821 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21342888-56e6-4a73-aaf3-cb3dbac94ba3 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:05.378533 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc7092-9eed-4acc-9885-83f5e8335c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:05.379451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc7092-9eed-4acc-9885-83f5e8335c3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:05.379747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc7092-9eed-4acc-9885-83f5e8335c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:05.380027 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc7092-9eed-4acc-9885-83f5e8335c3f admin admin] PATH_INFO: `/v3/projects/3a059c9b3ef14b6f92d7b76c6a738d0e/users/107848d4f55c4e5790b0dc6f9e0b69e5/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:05.395864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc7092-9eed-4acc-9885-83f5e8335c3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a059c9b3ef14b6f92d7b76c6a738d0e, user_id=107848d4f55c4e5790b0dc6f9e0b69e5, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:05.398819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc7092-9eed-4acc-9885-83f5e8335c3f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:05.500529 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1057/2150] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:10:05 2022] PUT /identity/v3/projects/3a059c9b3ef14b6f92d7b76c6a738d0e/users/107848d4f55c4e5790b0dc6f9e0b69e5/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:05.512574 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccf5d7-8a7e-4ee2-af10-3ea60b74232a None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:05.512876 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccf5d7-8a7e-4ee2-af10-3ea60b74232a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:05.513162 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccf5d7-8a7e-4ee2-af10-3ea60b74232a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:05.534886 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:05.545404 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-94ccf5d7-8a7e-4ee2-af10-3ea60b74232a None None] MFA Rules not processed for user `107848d4f55c4e5790b0dc6f9e0b69e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:05.546637 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-94ccf5d7-8a7e-4ee2-af10-3ea60b74232a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:05.615868 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1094/2151] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:05.628741 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc7092-9eed-4acc-9885-83f5e8335c3f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:05.631282 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-35dc7092-9eed-4acc-9885-83f5e8335c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:05.686256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:05.702533 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc7092-9eed-4acc-9885-83f5e8335c3f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:05.707896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680fc81-bc12-49da-9be9-c911529433fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:05.708672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680fc81-bc12-49da-9be9-c911529433fc service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:05.708981 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680fc81-bc12-49da-9be9-c911529433fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:05.709289 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680fc81-bc12-49da-9be9-c911529433fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:05.727079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680fc81-bc12-49da-9be9-c911529433fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:05.728882 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680fc81-bc12-49da-9be9-c911529433fc service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:05.739269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1058/2152] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:10:14.130077 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ccf5d7-8a7e-4ee2-af10-3ea60b74232a None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:14.256178 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ccf5d7-8a7e-4ee2-af10-3ea60b74232a None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:14.267632 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d0d90-e5c3-4d27-8818-4ba66c96e9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:14.268823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d0d90-e5c3-4d27-8818-4ba66c96e9eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:14.269596 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d0d90-e5c3-4d27-8818-4ba66c96e9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:14.269880 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d0d90-e5c3-4d27-8818-4ba66c96e9eb admin admin] PATH_INFO: `/v3/users/107848d4f55c4e5790b0dc6f9e0b69e5` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:14.272251 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d0d90-e5c3-4d27-8818-4ba66c96e9eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=107848d4f55c4e5790b0dc6f9e0b69e5)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:14.275233 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d0d90-e5c3-4d27-8818-4ba66c96e9eb admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:14.461506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7b8d0d90-e5c3-4d27-8818-4ba66c96e9eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '107848d4f55c4e5790b0dc6f9e0b69e5'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:14.462191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7b8d0d90-e5c3-4d27-8818-4ba66c96e9eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '107848d4f55c4e5790b0dc6f9e0b69e5'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:14.474503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7b8d0d90-e5c3-4d27-8818-4ba66c96e9eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '107848d4f55c4e5790b0dc6f9e0b69e5'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:14.475234 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7b8d0d90-e5c3-4d27-8818-4ba66c96e9eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '107848d4f55c4e5790b0dc6f9e0b69e5'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:14.482894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7b8d0d90-e5c3-4d27-8818-4ba66c96e9eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '107848d4f55c4e5790b0dc6f9e0b69e5'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:14.572681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1095/2153] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:10:14 2022] DELETE /identity/v3/users/107848d4f55c4e5790b0dc6f9e0b69e5 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:14.577616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccc704-a691-4383-8a8f-2a9a72807ac5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:14.578118 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccc704-a691-4383-8a8f-2a9a72807ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:14.578311 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccc704-a691-4383-8a8f-2a9a72807ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:14.601559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-16ccc704-a691-4383-8a8f-2a9a72807ac5 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:14.604027 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-16ccc704-a691-4383-8a8f-2a9a72807ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:14.673258 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1059/2154] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:14.682840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d0d90-e5c3-4d27-8818-4ba66c96e9eb admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:14.685122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7b8d0d90-e5c3-4d27-8818-4ba66c96e9eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:14.709000 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:14.753247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d0d90-e5c3-4d27-8818-4ba66c96e9eb admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:14.757122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe838d-ae69-4c7a-86dc-b2fe968dab89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:14.757815 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe838d-ae69-4c7a-86dc-b2fe968dab89 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:14.760209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe838d-ae69-4c7a-86dc-b2fe968dab89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:14.760209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe838d-ae69-4c7a-86dc-b2fe968dab89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:14.769180 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe838d-ae69-4c7a-86dc-b2fe968dab89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:14.770641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe838d-ae69-4c7a-86dc-b2fe968dab89 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:14.781098 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1096/2155] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:10:15.249803 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccc704-a691-4383-8a8f-2a9a72807ac5 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:15.251371 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-16ccc704-a691-4383-8a8f-2a9a72807ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:15.276120 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccc704-a691-4383-8a8f-2a9a72807ac5 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:15.288351 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3c38b-3f7e-4fab-85d1-59401665822e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:15.291811 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3c38b-3f7e-4fab-85d1-59401665822e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:15.295533 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3c38b-3f7e-4fab-85d1-59401665822e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:15.295533 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3c38b-3f7e-4fab-85d1-59401665822e admin admin] PATH_INFO: `/v3/projects/3a059c9b3ef14b6f92d7b76c6a738d0e` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:15.298212 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3c38b-3f7e-4fab-85d1-59401665822e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a059c9b3ef14b6f92d7b76c6a738d0e)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:15.302294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3c38b-3f7e-4fab-85d1-59401665822e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:15.474617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e4d3c38b-3f7e-4fab-85d1-59401665822e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a059c9b3ef14b6f92d7b76c6a738d0e', 'request_id': 'req-e4d3c38b-3f7e-4fab-85d1-59401665822e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:15.716838 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e4d3c38b-3f7e-4fab-85d1-59401665822e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a059c9b3ef14b6f92d7b76c6a738d0e', 'request_id': 'req-e4d3c38b-3f7e-4fab-85d1-59401665822e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:15.734391 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e4d3c38b-3f7e-4fab-85d1-59401665822e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a059c9b3ef14b6f92d7b76c6a738d0e', 'request_id': 'req-e4d3c38b-3f7e-4fab-85d1-59401665822e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:15.745152 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe838d-ae69-4c7a-86dc-b2fe968dab89 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:15.746602 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-54fe838d-ae69-4c7a-86dc-b2fe968dab89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:15.748497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e4d3c38b-3f7e-4fab-85d1-59401665822e admin admin] The token cache is being invalidate because project 3a059c9b3ef14b6f92d7b76c6a738d0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:10:15.753631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e4d3c38b-3f7e-4fab-85d1-59401665822e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:15.753631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e4d3c38b-3f7e-4fab-85d1-59401665822e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:15.753631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1060/2156] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:10:15 2022] DELETE /identity/v3/projects/3a059c9b3ef14b6f92d7b76c6a738d0e => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:15.760517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe838d-ae69-4c7a-86dc-b2fe968dab89 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:15.840252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee24b7-3524-4bcb-ace2-eaf37ab39c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:15.840252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee24b7-3524-4bcb-ace2-eaf37ab39c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:15.840252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee24b7-3524-4bcb-ace2-eaf37ab39c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:15.840252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee24b7-3524-4bcb-ace2-eaf37ab39c76 admin admin] PATH_INFO: `/v3/users/c7d4274811554f27a25a9ce786d7f802` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:15.841152 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee24b7-3524-4bcb-ace2-eaf37ab39c76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7d4274811554f27a25a9ce786d7f802)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:15.841152 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee24b7-3524-4bcb-ace2-eaf37ab39c76 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:16.124964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cbee24b7-3524-4bcb-ace2-eaf37ab39c76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7d4274811554f27a25a9ce786d7f802'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:16.125511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cbee24b7-3524-4bcb-ace2-eaf37ab39c76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7d4274811554f27a25a9ce786d7f802'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:16.137252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cbee24b7-3524-4bcb-ace2-eaf37ab39c76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7d4274811554f27a25a9ce786d7f802'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:16.138436 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cbee24b7-3524-4bcb-ace2-eaf37ab39c76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7d4274811554f27a25a9ce786d7f802'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:16.145372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cbee24b7-3524-4bcb-ace2-eaf37ab39c76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7d4274811554f27a25a9ce786d7f802'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:16.235343 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1097/2157] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:10:15 2022] DELETE /identity/v3/users/c7d4274811554f27a25a9ce786d7f802 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:10:16.240198 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513480f-1b80-45d5-af55-f6bd1d70d79b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:16.240361 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513480f-1b80-45d5-af55-f6bd1d70d79b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:16.240632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513480f-1b80-45d5-af55-f6bd1d70d79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:16.259395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-2513480f-1b80-45d5-af55-f6bd1d70d79b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:16.263920 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2513480f-1b80-45d5-af55-f6bd1d70d79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:16.312803 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:16.324083 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1061/2158] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:16.335394 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee24b7-3524-4bcb-ace2-eaf37ab39c76 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:16.338240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cbee24b7-3524-4bcb-ace2-eaf37ab39c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:16.404091 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:16.418579 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee24b7-3524-4bcb-ace2-eaf37ab39c76 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:16.423060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7175e7a-729f-496a-90ad-f3db11e803cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:16.423766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7175e7a-729f-496a-90ad-f3db11e803cf service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:16.425363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7175e7a-729f-496a-90ad-f3db11e803cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:16.425363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7175e7a-729f-496a-90ad-f3db11e803cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:16.439483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7175e7a-729f-496a-90ad-f3db11e803cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:16.441105 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7175e7a-729f-496a-90ad-f3db11e803cf service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:16.454539 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1098/2159] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:10:16.908423 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513480f-1b80-45d5-af55-f6bd1d70d79b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:16.911150 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2513480f-1b80-45d5-af55-f6bd1d70d79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:16.925821 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513480f-1b80-45d5-af55-f6bd1d70d79b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:16.930406 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ecd2d-f192-4694-a48c-2cdb1f11a8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:16.931414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ecd2d-f192-4694-a48c-2cdb1f11a8e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:16.931698 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ecd2d-f192-4694-a48c-2cdb1f11a8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:16.931969 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ecd2d-f192-4694-a48c-2cdb1f11a8e3 admin admin] PATH_INFO: `/v3/users/585f311a9cfa46bebc7baa787a111d83` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:16.933228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ecd2d-f192-4694-a48c-2cdb1f11a8e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=585f311a9cfa46bebc7baa787a111d83)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:16.935707 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ecd2d-f192-4694-a48c-2cdb1f11a8e3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:16.997871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7175e7a-729f-496a-90ad-f3db11e803cf service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:16.999532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a7175e7a-729f-496a-90ad-f3db11e803cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:17.015576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7175e7a-729f-496a-90ad-f3db11e803cf service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:17.020675 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41440ea1-e2b5-478b-830b-8143faa2c648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:17.021568 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41440ea1-e2b5-478b-830b-8143faa2c648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:17.021834 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41440ea1-e2b5-478b-830b-8143faa2c648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:17.022113 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41440ea1-e2b5-478b-830b-8143faa2c648 admin admin] PATH_INFO: `/v3/projects/f4219e8f4ef34bf9a89a313b2e044f67` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:17.023633 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41440ea1-e2b5-478b-830b-8143faa2c648 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4219e8f4ef34bf9a89a313b2e044f67)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:17.025638 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41440ea1-e2b5-478b-830b-8143faa2c648 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:17.145730 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-455ecd2d-f192-4694-a48c-2cdb1f11a8e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '585f311a9cfa46bebc7baa787a111d83'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:17.153737 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-455ecd2d-f192-4694-a48c-2cdb1f11a8e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '585f311a9cfa46bebc7baa787a111d83'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:17.154447 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-455ecd2d-f192-4694-a48c-2cdb1f11a8e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '585f311a9cfa46bebc7baa787a111d83'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:17.155072 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-455ecd2d-f192-4694-a48c-2cdb1f11a8e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '585f311a9cfa46bebc7baa787a111d83'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:17.168369 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-455ecd2d-f192-4694-a48c-2cdb1f11a8e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '585f311a9cfa46bebc7baa787a111d83'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:17.180089 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-41440ea1-e2b5-478b-830b-8143faa2c648 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4219e8f4ef34bf9a89a313b2e044f67', 'request_id': 'req-41440ea1-e2b5-478b-830b-8143faa2c648'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:17.264960 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1062/2160] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:10:16 2022] DELETE /identity/v3/users/585f311a9cfa46bebc7baa787a111d83 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:17.269566 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d16894-99cf-48cb-9d38-4d489ee0de45 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:17.269867 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d16894-99cf-48cb-9d38-4d489ee0de45 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:17.270153 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d16894-99cf-48cb-9d38-4d489ee0de45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:17.288681 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a8d16894-99cf-48cb-9d38-4d489ee0de45 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:17.289923 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a8d16894-99cf-48cb-9d38-4d489ee0de45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:17.321074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-41440ea1-e2b5-478b-830b-8143faa2c648 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4219e8f4ef34bf9a89a313b2e044f67', 'request_id': 'req-41440ea1-e2b5-478b-830b-8143faa2c648'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:17.328571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-41440ea1-e2b5-478b-830b-8143faa2c648 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4219e8f4ef34bf9a89a313b2e044f67', 'request_id': 'req-41440ea1-e2b5-478b-830b-8143faa2c648'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:17.340217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-41440ea1-e2b5-478b-830b-8143faa2c648 admin admin] The token cache is being invalidate because project f4219e8f4ef34bf9a89a313b2e044f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:10:17.340591 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-41440ea1-e2b5-478b-830b-8143faa2c648 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:17.341302 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-41440ea1-e2b5-478b-830b-8143faa2c648 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:17.343440 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1099/2161] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:10:16 2022] DELETE /identity/v3/projects/f4219e8f4ef34bf9a89a313b2e044f67 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:10:17.347050 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41440ea1-e2b5-478b-830b-8143faa2c648 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:17.348857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-41440ea1-e2b5-478b-830b-8143faa2c648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:17.361446 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1063/2162] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:17.369413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d16894-99cf-48cb-9d38-4d489ee0de45 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:17.370818 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a8d16894-99cf-48cb-9d38-4d489ee0de45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:17.440422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41440ea1-e2b5-478b-830b-8143faa2c648 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:17.443800 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955c346-1b47-4329-acf9-47f22b650bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:17.445086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955c346-1b47-4329-acf9-47f22b650bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:17.445421 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955c346-1b47-4329-acf9-47f22b650bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:17.445659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955c346-1b47-4329-acf9-47f22b650bb4 admin admin] PATH_INFO: `/v3/users/ce346b6a087c43009f1926240f211717` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:17.447035 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955c346-1b47-4329-acf9-47f22b650bb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce346b6a087c43009f1926240f211717)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:17.450845 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955c346-1b47-4329-acf9-47f22b650bb4 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:17.461539 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d16894-99cf-48cb-9d38-4d489ee0de45 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:17.465812 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c21cc1-8c6f-4153-8ec3-81b046f9273c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:17.466493 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c21cc1-8c6f-4153-8ec3-81b046f9273c service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:17.466788 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c21cc1-8c6f-4153-8ec3-81b046f9273c service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:17.467068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c21cc1-8c6f-4153-8ec3-81b046f9273c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:17.478194 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c21cc1-8c6f-4153-8ec3-81b046f9273c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:17.479773 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c21cc1-8c6f-4153-8ec3-81b046f9273c service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:17.490326 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1064/2163] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:10:17.723319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5955c346-1b47-4329-acf9-47f22b650bb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce346b6a087c43009f1926240f211717'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:17.723698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5955c346-1b47-4329-acf9-47f22b650bb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce346b6a087c43009f1926240f211717'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:17.737891 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5955c346-1b47-4329-acf9-47f22b650bb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce346b6a087c43009f1926240f211717'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:17.739759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5955c346-1b47-4329-acf9-47f22b650bb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce346b6a087c43009f1926240f211717'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:17.745092 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:17.748735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5955c346-1b47-4329-acf9-47f22b650bb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce346b6a087c43009f1926240f211717'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:17.814212 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1100/2164] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:10:17 2022] DELETE /identity/v3/users/ce346b6a087c43009f1926240f211717 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:17.993047 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c21cc1-8c6f-4153-8ec3-81b046f9273c service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:17.995137 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a1c21cc1-8c6f-4153-8ec3-81b046f9273c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:18.009778 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c21cc1-8c6f-4153-8ec3-81b046f9273c service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:18.025057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:18.118824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f16772-5a1a-44ed-8c9d-8846e5ee6482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:18.120114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f16772-5a1a-44ed-8c9d-8846e5ee6482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:18.120416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f16772-5a1a-44ed-8c9d-8846e5ee6482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:18.121284 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f16772-5a1a-44ed-8c9d-8846e5ee6482 admin admin] PATH_INFO: `/v3/projects/367d424ad7094fc7906f513627a95334` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:18.124120 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f16772-5a1a-44ed-8c9d-8846e5ee6482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=367d424ad7094fc7906f513627a95334)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:18.126102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f16772-5a1a-44ed-8c9d-8846e5ee6482 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:18.241181 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-69f16772-5a1a-44ed-8c9d-8846e5ee6482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '367d424ad7094fc7906f513627a95334', 'request_id': 'req-69f16772-5a1a-44ed-8c9d-8846e5ee6482'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:18.323022 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955c346-1b47-4329-acf9-47f22b650bb4 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:18.324651 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5955c346-1b47-4329-acf9-47f22b650bb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:18.326561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-69f16772-5a1a-44ed-8c9d-8846e5ee6482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '367d424ad7094fc7906f513627a95334', 'request_id': 'req-69f16772-5a1a-44ed-8c9d-8846e5ee6482'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:18.334752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-69f16772-5a1a-44ed-8c9d-8846e5ee6482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '367d424ad7094fc7906f513627a95334', 'request_id': 'req-69f16772-5a1a-44ed-8c9d-8846e5ee6482'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:18.339574 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955c346-1b47-4329-acf9-47f22b650bb4 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:18.348511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-69f16772-5a1a-44ed-8c9d-8846e5ee6482 admin admin] The token cache is being invalidate because project 367d424ad7094fc7906f513627a95334 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:10:18.348836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-69f16772-5a1a-44ed-8c9d-8846e5ee6482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:18.349499 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-69f16772-5a1a-44ed-8c9d-8846e5ee6482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:18.351628 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1065/2165] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:10:17 2022] DELETE /identity/v3/projects/367d424ad7094fc7906f513627a95334 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:18.357381 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65253563-fbd8-4c4f-9fb1-4e0d73143162 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:18.357744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65253563-fbd8-4c4f-9fb1-4e0d73143162 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:18.358050 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65253563-fbd8-4c4f-9fb1-4e0d73143162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:18.377774 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-65253563-fbd8-4c4f-9fb1-4e0d73143162 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:18.384407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-65253563-fbd8-4c4f-9fb1-4e0d73143162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:18.420159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6d69b-42ae-43bd-a78a-bfeeb728ee70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:18.421015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6d69b-42ae-43bd-a78a-bfeeb728ee70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:18.421699 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6d69b-42ae-43bd-a78a-bfeeb728ee70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:18.421699 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6d69b-42ae-43bd-a78a-bfeeb728ee70 admin admin] PATH_INFO: `/v3/projects/9665ba042f394c018cfb4b597da226bf` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:18.422994 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc6d69b-42ae-43bd-a78a-bfeeb728ee70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9665ba042f394c018cfb4b597da226bf)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:18.425255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc6d69b-42ae-43bd-a78a-bfeeb728ee70 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:18.461017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1066/2166] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:18.468701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc1f2d-588a-4c74-a993-22c71910c014 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:18.468989 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc1f2d-588a-4c74-a993-22c71910c014 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:18.469269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc1f2d-588a-4c74-a993-22c71910c014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:18.491553 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a6bc1f2d-588a-4c74-a993-22c71910c014 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:18.491553 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a6bc1f2d-588a-4c74-a993-22c71910c014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:18.503522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1067/2167] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:18.507883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc1f2d-588a-4c74-a993-22c71910c014 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:18.517257 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc1f2d-588a-4c74-a993-22c71910c014 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:18.522748 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336a56f-7566-4691-ab9b-9b4ecb0b115c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:18.523341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336a56f-7566-4691-ab9b-9b4ecb0b115c admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:18.523654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336a56f-7566-4691-ab9b-9b4ecb0b115c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:18.523920 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336a56f-7566-4691-ab9b-9b4ecb0b115c admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:18.525633 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336a56f-7566-4691-ab9b-9b4ecb0b115c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:10:18.525950 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336a56f-7566-4691-ab9b-9b4ecb0b115c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:18.527695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336a56f-7566-4691-ab9b-9b4ecb0b115c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:18.542167 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1068/2168] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 19:10:18.546851 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336a56f-7566-4691-ab9b-9b4ecb0b115c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:18.557116 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336a56f-7566-4691-ab9b-9b4ecb0b115c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:18.558151 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bfc6d69b-42ae-43bd-a78a-bfeeb728ee70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9665ba042f394c018cfb4b597da226bf', 'request_id': 'req-bfc6d69b-42ae-43bd-a78a-bfeeb728ee70'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:18.577614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:18.616072 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebe7b2-5b3a-4d82-bc73-b3e6e6b50d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:18.616695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebe7b2-5b3a-4d82-bc73-b3e6e6b50d2a admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:18.617124 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebe7b2-5b3a-4d82-bc73-b3e6e6b50d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:18.617416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebe7b2-5b3a-4d82-bc73-b3e6e6b50d2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:18.618121 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebe7b2-5b3a-4d82-bc73-b3e6e6b50d2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:18.620214 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebe7b2-5b3a-4d82-bc73-b3e6e6b50d2a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:18.629152 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bfc6d69b-42ae-43bd-a78a-bfeeb728ee70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9665ba042f394c018cfb4b597da226bf', 'request_id': 'req-bfc6d69b-42ae-43bd-a78a-bfeeb728ee70'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:18.635739 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bfc6d69b-42ae-43bd-a78a-bfeeb728ee70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9665ba042f394c018cfb4b597da226bf', 'request_id': 'req-bfc6d69b-42ae-43bd-a78a-bfeeb728ee70'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:18.648169 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bfc6d69b-42ae-43bd-a78a-bfeeb728ee70 admin admin] The token cache is being invalidate because project 9665ba042f394c018cfb4b597da226bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:10:18.648460 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bfc6d69b-42ae-43bd-a78a-bfeeb728ee70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:18.649134 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bfc6d69b-42ae-43bd-a78a-bfeeb728ee70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:18.650713 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1101/2169] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:10:18 2022] DELETE /identity/v3/projects/9665ba042f394c018cfb4b597da226bf => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:18.656004 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78f5df-986e-48df-84eb-323e172bc48c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:18.656299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78f5df-986e-48df-84eb-323e172bc48c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:18.656589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78f5df-986e-48df-84eb-323e172bc48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:18.675697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-fb78f5df-986e-48df-84eb-323e172bc48c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:18.676953 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-fb78f5df-986e-48df-84eb-323e172bc48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:18.686011 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1102/2170] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:18.695667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c40164-c057-458b-a09b-1ea700316107 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:18.696304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c40164-c057-458b-a09b-1ea700316107 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:18.701047 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c40164-c057-458b-a09b-1ea700316107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:18.718477 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-b3c40164-c057-458b-a09b-1ea700316107 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:18.719782 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b3c40164-c057-458b-a09b-1ea700316107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:18.731194 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1069/2171] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:10:18 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:10:18.731399 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1103/2172] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:18.732382 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebe7b2-5b3a-4d82-bc73-b3e6e6b50d2a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:18.732771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c40164-c057-458b-a09b-1ea700316107 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:18.735268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-53ebe7b2-5b3a-4d82-bc73-b3e6e6b50d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:18.742047 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c40164-c057-458b-a09b-1ea700316107 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:18.799155 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9bd09-d1a2-40c9-88ca-dd7155821218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:18.799962 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9bd09-d1a2-40c9-88ca-dd7155821218 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:18.800403 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9bd09-d1a2-40c9-88ca-dd7155821218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:18.800691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9bd09-d1a2-40c9-88ca-dd7155821218 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:18.801428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9bd09-d1a2-40c9-88ca-dd7155821218 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:10:18.801743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9bd09-d1a2-40c9-88ca-dd7155821218 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:18.803403 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9bd09-d1a2-40c9-88ca-dd7155821218 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:18.803907 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebe7b2-5b3a-4d82-bc73-b3e6e6b50d2a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:18.808952 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109546c-8f58-406e-8e41-18f4b9727f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:18.810280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109546c-8f58-406e-8e41-18f4b9727f98 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:18.810582 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109546c-8f58-406e-8e41-18f4b9727f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:18.810814 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:18.810940 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109546c-8f58-406e-8e41-18f4b9727f98 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:18.811615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109546c-8f58-406e-8e41-18f4b9727f98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:18.813726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109546c-8f58-406e-8e41-18f4b9727f98 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:18.817173 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1104/2173] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 19:10:18.820737 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9bd09-d1a2-40c9-88ca-dd7155821218 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:18.832204 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9bd09-d1a2-40c9-88ca-dd7155821218 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:18.837256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6524a8e-f303-495c-87f4-11d0b891bf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:18.837937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6524a8e-f303-495c-87f4-11d0b891bf33 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:18.838223 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6524a8e-f303-495c-87f4-11d0b891bf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:18.838526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6524a8e-f303-495c-87f4-11d0b891bf33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:18.839263 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6524a8e-f303-495c-87f4-11d0b891bf33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:18.841307 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6524a8e-f303-495c-87f4-11d0b891bf33 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:18.944761 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1070/2174] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:10:18 2022] POST /identity/v3/users => generated 287 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:10:18.948817 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109546c-8f58-406e-8e41-18f4b9727f98 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:18.957971 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1105/2175] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:10:18 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:10:18.959524 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109546c-8f58-406e-8e41-18f4b9727f98 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:18.962163 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6524a8e-f303-495c-87f4-11d0b891bf33 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:19.028488 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c0d02-74ea-4c3d-850f-dad42a430172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:19.029168 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c0d02-74ea-4c3d-850f-dad42a430172 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:19.029454 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c0d02-74ea-4c3d-850f-dad42a430172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:19.029733 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c0d02-74ea-4c3d-850f-dad42a430172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:19.030465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c0d02-74ea-4c3d-850f-dad42a430172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:19.032092 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c0d02-74ea-4c3d-850f-dad42a430172 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:19.041228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6524a8e-f303-495c-87f4-11d0b891bf33 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:19.042406 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1071/2176] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:10:19.046842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c0d02-74ea-4c3d-850f-dad42a430172 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:19.049539 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ac5a3-b3c7-4e94-9a42-e2129e2d2cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:19.050222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ac5a3-b3c7-4e94-9a42-e2129e2d2cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:19.050512 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ac5a3-b3c7-4e94-9a42-e2129e2d2cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:19.050789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ac5a3-b3c7-4e94-9a42-e2129e2d2cf9 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:19.051813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ac5a3-b3c7-4e94-9a42-e2129e2d2cf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:19.054491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ac5a3-b3c7-4e94-9a42-e2129e2d2cf9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:19.056508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c0d02-74ea-4c3d-850f-dad42a430172 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:19.061122 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610bc686-b14f-428a-88f3-8d55e47e42ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:19.061764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610bc686-b14f-428a-88f3-8d55e47e42ec admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:19.062051 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610bc686-b14f-428a-88f3-8d55e47e42ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:19.062330 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610bc686-b14f-428a-88f3-8d55e47e42ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:19.063053 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610bc686-b14f-428a-88f3-8d55e47e42ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:19.064690 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610bc686-b14f-428a-88f3-8d55e47e42ec admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:19.076058 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1072/2177] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:10:19.080218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610bc686-b14f-428a-88f3-8d55e47e42ec admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:19.090104 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610bc686-b14f-428a-88f3-8d55e47e42ec admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:19.094824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46c3de-99fa-4b3b-a46b-7360435e442b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:19.095967 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46c3de-99fa-4b3b-a46b-7360435e442b admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:19.100082 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46c3de-99fa-4b3b-a46b-7360435e442b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:19.100082 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46c3de-99fa-4b3b-a46b-7360435e442b admin admin] PATH_INFO: `/v3/projects/66ff5513df764ccbb073b2aaf8fb3336/users/98ea4d7dc953432ba9040b6d4b35f107/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:19.113524 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46c3de-99fa-4b3b-a46b-7360435e442b admin admin] RBAC: Authorizing `identity:create_grant(project_id=66ff5513df764ccbb073b2aaf8fb3336, user_id=98ea4d7dc953432ba9040b6d4b35f107, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:19.116147 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1106/2178] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:10:18 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:10:19.117535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46c3de-99fa-4b3b-a46b-7360435e442b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:19.121348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ac5a3-b3c7-4e94-9a42-e2129e2d2cf9 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:19.134249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ac5a3-b3c7-4e94-9a42-e2129e2d2cf9 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:19.139123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396e27a-8fc9-47ff-94dc-416032c65ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:19.139857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396e27a-8fc9-47ff-94dc-416032c65ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:19.140171 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396e27a-8fc9-47ff-94dc-416032c65ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:19.140448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396e27a-8fc9-47ff-94dc-416032c65ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:19.141176 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396e27a-8fc9-47ff-94dc-416032c65ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:19.142778 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396e27a-8fc9-47ff-94dc-416032c65ea2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:19.153578 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1107/2179] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:10:19.157735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396e27a-8fc9-47ff-94dc-416032c65ea2 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:19.167268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396e27a-8fc9-47ff-94dc-416032c65ea2 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:19.172517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02b413-82ff-4915-af34-cacc6cbd99b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:19.173165 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02b413-82ff-4915-af34-cacc6cbd99b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:19.173454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02b413-82ff-4915-af34-cacc6cbd99b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:19.173734 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02b413-82ff-4915-af34-cacc6cbd99b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:19.174446 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02b413-82ff-4915-af34-cacc6cbd99b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:19.176384 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02b413-82ff-4915-af34-cacc6cbd99b7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:19.185394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:19.187415 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1108/2180] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:10:19.191656 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02b413-82ff-4915-af34-cacc6cbd99b7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:19.194158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1073/2181] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:10:19 2022] PUT /identity/v3/projects/66ff5513df764ccbb073b2aaf8fb3336/users/98ea4d7dc953432ba9040b6d4b35f107/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:19.205468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cd1a3-157e-4545-829e-2ce0bebd1953 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:19.206393 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cd1a3-157e-4545-829e-2ce0bebd1953 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:19.206741 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cd1a3-157e-4545-829e-2ce0bebd1953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:19.210321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02b413-82ff-4915-af34-cacc6cbd99b7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:19.237915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-fd0cd1a3-157e-4545-829e-2ce0bebd1953 None None] MFA Rules not processed for user `98ea4d7dc953432ba9040b6d4b35f107`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:19.239546 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-fd0cd1a3-157e-4545-829e-2ce0bebd1953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:19.287844 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128557a-98cc-4390-abc6-ede645fe9bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:19.289136 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128557a-98cc-4390-abc6-ede645fe9bcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:19.289136 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128557a-98cc-4390-abc6-ede645fe9bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:19.289343 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128557a-98cc-4390-abc6-ede645fe9bcb admin admin] PATH_INFO: `/v3/projects/4d4ec30922c44074bcd90beea3ca7f59/users/42ae67ee06be4a0ca41cc6442d837d8c/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:19.312799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128557a-98cc-4390-abc6-ede645fe9bcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d4ec30922c44074bcd90beea3ca7f59, user_id=42ae67ee06be4a0ca41cc6442d837d8c, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:19.315615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128557a-98cc-4390-abc6-ede645fe9bcb admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:19.334923 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1074/2182] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:19.341658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cd1a3-157e-4545-829e-2ce0bebd1953 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:19.358144 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cd1a3-157e-4545-829e-2ce0bebd1953 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:19.364087 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a2753-8407-4fe9-93db-175fd050dd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:19.365228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a2753-8407-4fe9-93db-175fd050dd47 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:19.365620 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a2753-8407-4fe9-93db-175fd050dd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:19.366010 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a2753-8407-4fe9-93db-175fd050dd47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:19.366732 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a2753-8407-4fe9-93db-175fd050dd47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:19.368740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a2753-8407-4fe9-93db-175fd050dd47 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:19.415694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1109/2183] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:10:19 2022] PUT /identity/v3/projects/4d4ec30922c44074bcd90beea3ca7f59/users/42ae67ee06be4a0ca41cc6442d837d8c/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:19.428516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd704cb2-7359-4dbc-86cd-d1a0e9003b05 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:19.428807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd704cb2-7359-4dbc-86cd-d1a0e9003b05 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:19.429087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd704cb2-7359-4dbc-86cd-d1a0e9003b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:19.456384 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-bd704cb2-7359-4dbc-86cd-d1a0e9003b05 None None] MFA Rules not processed for user `42ae67ee06be4a0ca41cc6442d837d8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:19.457516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bd704cb2-7359-4dbc-86cd-d1a0e9003b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:19.470648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:19.510498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1075/2184] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:10:19 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:10:19.516425 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a2753-8407-4fe9-93db-175fd050dd47 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:19.541553 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1110/2185] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:19.547708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd704cb2-7359-4dbc-86cd-d1a0e9003b05 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:19.612380 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a2753-8407-4fe9-93db-175fd050dd47 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:19.617667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf20be-6800-4daa-953e-e85f3a102d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:19.618356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf20be-6800-4daa-953e-e85f3a102d0f admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:19.618690 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf20be-6800-4daa-953e-e85f3a102d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:19.618970 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf20be-6800-4daa-953e-e85f3a102d0f admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:19.619827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf20be-6800-4daa-953e-e85f3a102d0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:19.621815 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf20be-6800-4daa-953e-e85f3a102d0f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:19.634916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd704cb2-7359-4dbc-86cd-d1a0e9003b05 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:19.639790 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2840472-13a0-4274-9fa6-7b96182872de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:19.640474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2840472-13a0-4274-9fa6-7b96182872de admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:19.640764 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2840472-13a0-4274-9fa6-7b96182872de admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:19.641049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2840472-13a0-4274-9fa6-7b96182872de admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:19.641774 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2840472-13a0-4274-9fa6-7b96182872de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:19.643829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2840472-13a0-4274-9fa6-7b96182872de admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:19.712196 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1076/2186] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:10:19 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:10:19.716224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf20be-6800-4daa-953e-e85f3a102d0f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:19.726475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf20be-6800-4daa-953e-e85f3a102d0f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:19.731847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81ecde-754e-4472-91f4-61a28fbad9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:19.732534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81ecde-754e-4472-91f4-61a28fbad9af admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:19.732809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81ecde-754e-4472-91f4-61a28fbad9af admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:19.733085 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81ecde-754e-4472-91f4-61a28fbad9af admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:19.733821 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81ecde-754e-4472-91f4-61a28fbad9af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:19.735455 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81ecde-754e-4472-91f4-61a28fbad9af admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:19.743976 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1111/2187] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:10:19 2022] POST /identity/v3/projects => generated 379 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:10:19.745004 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2840472-13a0-4274-9fa6-7b96182872de admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:19.749987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1077/2188] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:10:19.751329 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81ecde-754e-4472-91f4-61a28fbad9af admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:19.761429 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81ecde-754e-4472-91f4-61a28fbad9af admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:19.808717 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:19.817460 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2840472-13a0-4274-9fa6-7b96182872de admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:19.822495 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576b48c-426a-4364-bb28-86206684f80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:19.823217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576b48c-426a-4364-bb28-86206684f80f admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:19.827050 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576b48c-426a-4364-bb28-86206684f80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:19.827050 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576b48c-426a-4364-bb28-86206684f80f admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:19.827050 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576b48c-426a-4364-bb28-86206684f80f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:19.827050 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576b48c-426a-4364-bb28-86206684f80f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:19.827551 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40f0c5-5ebe-45ba-b964-50dc22351dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:19.827551 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40f0c5-5ebe-45ba-b964-50dc22351dda admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:19.827947 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40f0c5-5ebe-45ba-b964-50dc22351dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:19.828243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40f0c5-5ebe-45ba-b964-50dc22351dda admin admin] PATH_INFO: `/v3/projects/62a13131a37f4d5f955b55889e12c2d7/users/228983ffa26f42449160eb76b1dce79d/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:19.845342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40f0c5-5ebe-45ba-b964-50dc22351dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a13131a37f4d5f955b55889e12c2d7, user_id=228983ffa26f42449160eb76b1dce79d, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:19.848738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40f0c5-5ebe-45ba-b964-50dc22351dda admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:19.944511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1112/2189] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:10:19 2022] POST /identity/v3/users => generated 296 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:10:19.948617 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576b48c-426a-4364-bb28-86206684f80f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:19.959426 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576b48c-426a-4364-bb28-86206684f80f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:19.962183 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1078/2190] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:10:19 2022] PUT /identity/v3/projects/62a13131a37f4d5f955b55889e12c2d7/users/228983ffa26f42449160eb76b1dce79d/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:19.966115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40f0c5-5ebe-45ba-b964-50dc22351dda admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:20.012359 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:20.046559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199612c-0f30-4a3f-b2a2-eb6376247361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:20.047311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199612c-0f30-4a3f-b2a2-eb6376247361 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:20.047600 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199612c-0f30-4a3f-b2a2-eb6376247361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:20.047902 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199612c-0f30-4a3f-b2a2-eb6376247361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:20.048687 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199612c-0f30-4a3f-b2a2-eb6376247361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:20.050309 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199612c-0f30-4a3f-b2a2-eb6376247361 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:20.053200 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40f0c5-5ebe-45ba-b964-50dc22351dda admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:20.058965 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4acfa-bcec-47ff-9781-1c0b1895e23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:20.059879 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4acfa-bcec-47ff-9781-1c0b1895e23d admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:20.060182 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4acfa-bcec-47ff-9781-1c0b1895e23d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:20.060458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4acfa-bcec-47ff-9781-1c0b1895e23d admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:20.061227 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1113/2191] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:10:20.061404 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4acfa-bcec-47ff-9781-1c0b1895e23d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:20.062901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4acfa-bcec-47ff-9781-1c0b1895e23d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:20.066252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199612c-0f30-4a3f-b2a2-eb6376247361 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:20.076448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1079/2192] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:10:20.081157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4acfa-bcec-47ff-9781-1c0b1895e23d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:20.085420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199612c-0f30-4a3f-b2a2-eb6376247361 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:20.092962 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4acfa-bcec-47ff-9781-1c0b1895e23d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:20.097878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd852c0-250c-462d-9f02-7962a53e7dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:20.098802 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd852c0-250c-462d-9f02-7962a53e7dc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:20.099108 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd852c0-250c-462d-9f02-7962a53e7dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:20.099399 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd852c0-250c-462d-9f02-7962a53e7dc2 admin admin] PATH_INFO: `/v3/projects/38f3e7b79c9c494b981aff1ca39e242a/users/1f5f5359f889475b90fdc29693eb4af4/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:20.100421 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660efc58-3cb2-4133-ab8a-5fe91b79c2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:20.101325 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660efc58-3cb2-4133-ab8a-5fe91b79c2dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:20.101739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660efc58-3cb2-4133-ab8a-5fe91b79c2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:20.102220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660efc58-3cb2-4133-ab8a-5fe91b79c2dc admin admin] PATH_INFO: `/v3/projects/62a13131a37f4d5f955b55889e12c2d7/users/228983ffa26f42449160eb76b1dce79d/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:20.104861 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660efc58-3cb2-4133-ab8a-5fe91b79c2dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a13131a37f4d5f955b55889e12c2d7, user_id=228983ffa26f42449160eb76b1dce79d, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:20.110216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660efc58-3cb2-4133-ab8a-5fe91b79c2dc admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:20.117688 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd852c0-250c-462d-9f02-7962a53e7dc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38f3e7b79c9c494b981aff1ca39e242a, user_id=1f5f5359f889475b90fdc29693eb4af4, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:20.120195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1080/2193] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:10:20 2022] PUT /identity/v3/projects/62a13131a37f4d5f955b55889e12c2d7/users/228983ffa26f42449160eb76b1dce79d/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:10:20.120810 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd852c0-250c-462d-9f02-7962a53e7dc2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:20.142172 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bed9a-fae3-4f80-81f4-58c090d2b12a None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:20.149675 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bed9a-fae3-4f80-81f4-58c090d2b12a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:20.149675 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bed9a-fae3-4f80-81f4-58c090d2b12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:20.171740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-448bed9a-fae3-4f80-81f4-58c090d2b12a None None] MFA Rules not processed for user `228983ffa26f42449160eb76b1dce79d`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:20.172957 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-448bed9a-fae3-4f80-81f4-58c090d2b12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:20.213109 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1114/2194] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:10:20 2022] PUT /identity/v3/projects/38f3e7b79c9c494b981aff1ca39e242a/users/1f5f5359f889475b90fdc29693eb4af4/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:10:20.217367 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd852c0-250c-462d-9f02-7962a53e7dc2 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:20.263045 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1081/2195] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:20.275440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bed9a-fae3-4f80-81f4-58c090d2b12a None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:20.277359 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-448bed9a-fae3-4f80-81f4-58c090d2b12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:20.301456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd852c0-250c-462d-9f02-7962a53e7dc2 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:20.306490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7017f-1e20-4ca8-99dc-4a37e9cd072e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:20.307182 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7017f-1e20-4ca8-99dc-4a37e9cd072e admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:20.307472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7017f-1e20-4ca8-99dc-4a37e9cd072e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:20.307749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7017f-1e20-4ca8-99dc-4a37e9cd072e admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:20.308488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7017f-1e20-4ca8-99dc-4a37e9cd072e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:20.310593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7017f-1e20-4ca8-99dc-4a37e9cd072e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:20.318977 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:20.321668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1115/2196] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:10:20.326980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7017f-1e20-4ca8-99dc-4a37e9cd072e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:20.343698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7017f-1e20-4ca8-99dc-4a37e9cd072e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:20.344388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bed9a-fae3-4f80-81f4-58c090d2b12a None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:20.351620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a15b0-0a99-4127-a6e1-1d5ca71bc308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:20.352595 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a15b0-0a99-4127-a6e1-1d5ca71bc308 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:20.352905 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a15b0-0a99-4127-a6e1-1d5ca71bc308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:20.353219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a15b0-0a99-4127-a6e1-1d5ca71bc308 admin admin] PATH_INFO: `/v3/projects/38f3e7b79c9c494b981aff1ca39e242a/users/1f5f5359f889475b90fdc29693eb4af4/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:20.355320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a15b0-0a99-4127-a6e1-1d5ca71bc308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38f3e7b79c9c494b981aff1ca39e242a, user_id=1f5f5359f889475b90fdc29693eb4af4, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:20.358065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a15b0-0a99-4127-a6e1-1d5ca71bc308 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:20.375263 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1116/2197] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:10:20 2022] PUT /identity/v3/projects/38f3e7b79c9c494b981aff1ca39e242a/users/1f5f5359f889475b90fdc29693eb4af4/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:10:20.381991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a4b6c-5243-41a5-98ba-c46aafb7d074 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:20.382290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a4b6c-5243-41a5-98ba-c46aafb7d074 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:20.382575 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a4b6c-5243-41a5-98ba-c46aafb7d074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:20.402368 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764d1c1-5151-4dca-abce-22c98337c02a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:20.403067 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764d1c1-5151-4dca-abce-22c98337c02a service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:20.403395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764d1c1-5151-4dca-abce-22c98337c02a service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:20.403679 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764d1c1-5151-4dca-abce-22c98337c02a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:20.414571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-944a4b6c-5243-41a5-98ba-c46aafb7d074 None None] MFA Rules not processed for user `1f5f5359f889475b90fdc29693eb4af4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:20.416062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-944a4b6c-5243-41a5-98ba-c46aafb7d074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:20.472573 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7764d1c1-5151-4dca-abce-22c98337c02a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:20.474091 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7764d1c1-5151-4dca-abce-22c98337c02a service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:20.484832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1082/2198] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:10:20.493057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1117/2199] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:20.505070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764d1c1-5151-4dca-abce-22c98337c02a service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:20.506482 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7764d1c1-5151-4dca-abce-22c98337c02a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:20.519090 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764d1c1-5151-4dca-abce-22c98337c02a service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:20.576611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4be38-4d38-47e7-8f02-6e4fee6b7002 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:20.577256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4be38-4d38-47e7-8f02-6e4fee6b7002 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:20.577541 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4be38-4d38-47e7-8f02-6e4fee6b7002 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:20.577820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4be38-4d38-47e7-8f02-6e4fee6b7002 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:20.588758 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4be38-4d38-47e7-8f02-6e4fee6b7002 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:20.590185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4be38-4d38-47e7-8f02-6e4fee6b7002 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:20.601343 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1083/2200] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:10:21.513794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a4b6c-5243-41a5-98ba-c46aafb7d074 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:21.523930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a4b6c-5243-41a5-98ba-c46aafb7d074 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:21.529045 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8fc02-03d8-48b3-bf38-d91ddf0a2fdf tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5f5359f889475b90fdc29693eb4af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38f3e7b79c9c494b981aff1ca39e242a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:21.529664 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8fc02-03d8-48b3-bf38-d91ddf0a2fdf tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:21.529955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8fc02-03d8-48b3-bf38-d91ddf0a2fdf tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:21.530232 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8fc02-03d8-48b3-bf38-d91ddf0a2fdf tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:21.531086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b8fc02-03d8-48b3-bf38-d91ddf0a2fdf tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:10:21.531395 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b8fc02-03d8-48b3-bf38-d91ddf0a2fdf tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:21.533023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b8fc02-03d8-48b3-bf38-d91ddf0a2fdf tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:21.539850 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:21.546197 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1118/2201] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:10:21.550467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4be38-4d38-47e7-8f02-6e4fee6b7002 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:21.560556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4be38-4d38-47e7-8f02-6e4fee6b7002 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:21.565636 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71488ba-7474-4a14-8cc1-e105dbfa80de tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5f5359f889475b90fdc29693eb4af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38f3e7b79c9c494b981aff1ca39e242a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:21.566303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71488ba-7474-4a14-8cc1-e105dbfa80de tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:21.566600 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71488ba-7474-4a14-8cc1-e105dbfa80de tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:21.566886 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71488ba-7474-4a14-8cc1-e105dbfa80de tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:21.567759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71488ba-7474-4a14-8cc1-e105dbfa80de tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:21.569781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71488ba-7474-4a14-8cc1-e105dbfa80de tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:21.701056 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1084/2202] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:10:21 2022] POST /identity/v3/projects => generated 355 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:10:21.705311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8fc02-03d8-48b3-bf38-d91ddf0a2fdf tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:21.717700 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8fc02-03d8-48b3-bf38-d91ddf0a2fdf tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:21.776796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02940c4c-e092-4076-a041-8241b5313053 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5f5359f889475b90fdc29693eb4af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38f3e7b79c9c494b981aff1ca39e242a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:21.777461 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02940c4c-e092-4076-a041-8241b5313053 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:21.778276 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02940c4c-e092-4076-a041-8241b5313053 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:21.778559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02940c4c-e092-4076-a041-8241b5313053 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:21.779375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02940c4c-e092-4076-a041-8241b5313053 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:10:21.779630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02940c4c-e092-4076-a041-8241b5313053 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:21.781491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02940c4c-e092-4076-a041-8241b5313053 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:21.792838 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1119/2203] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:10:21.830206 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71488ba-7474-4a14-8cc1-e105dbfa80de tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:21.841119 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71488ba-7474-4a14-8cc1-e105dbfa80de tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:21.847197 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53205e5-75ac-499a-9358-d703df8421f8 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5f5359f889475b90fdc29693eb4af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38f3e7b79c9c494b981aff1ca39e242a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:21.848100 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53205e5-75ac-499a-9358-d703df8421f8 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:21.848385 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53205e5-75ac-499a-9358-d703df8421f8 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:21.848663 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53205e5-75ac-499a-9358-d703df8421f8 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] PATH_INFO: `/v3/projects/a72006813c9341bc93ffaaec4580c791` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:21.849985 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53205e5-75ac-499a-9358-d703df8421f8 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a72006813c9341bc93ffaaec4580c791)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:21.852089 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53205e5-75ac-499a-9358-d703df8421f8 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:21.878066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:21.996569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b53205e5-75ac-499a-9358-d703df8421f8 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a72006813c9341bc93ffaaec4580c791', 'request_id': 'req-b53205e5-75ac-499a-9358-d703df8421f8'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:22.138634 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b53205e5-75ac-499a-9358-d703df8421f8 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a72006813c9341bc93ffaaec4580c791', 'request_id': 'req-b53205e5-75ac-499a-9358-d703df8421f8'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:22.146180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b53205e5-75ac-499a-9358-d703df8421f8 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a72006813c9341bc93ffaaec4580c791', 'request_id': 'req-b53205e5-75ac-499a-9358-d703df8421f8'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:22.160221 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b53205e5-75ac-499a-9358-d703df8421f8 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] The token cache is being invalidate because project a72006813c9341bc93ffaaec4580c791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:10:22.161142 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b53205e5-75ac-499a-9358-d703df8421f8 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:22.161934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b53205e5-75ac-499a-9358-d703df8421f8 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:22.163583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1085/2204] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:10:21 2022] DELETE /identity/v3/projects/a72006813c9341bc93ffaaec4580c791 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:10:22.816419 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02940c4c-e092-4076-a041-8241b5313053 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:22.818307 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-02940c4c-e092-4076-a041-8241b5313053 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:22.900975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02940c4c-e092-4076-a041-8241b5313053 tempest-VolumeQuotaClassesTest-953705142 tempest-VolumeQuotaClassesTest-953705142-project-admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:22.964979 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41a8dd-fc24-4ebf-8b64-0838fae3fdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:22.966453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41a8dd-fc24-4ebf-8b64-0838fae3fdf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:22.966735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41a8dd-fc24-4ebf-8b64-0838fae3fdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:22.967021 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41a8dd-fc24-4ebf-8b64-0838fae3fdf6 admin admin] PATH_INFO: `/v3/users/42ae67ee06be4a0ca41cc6442d837d8c` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:22.968301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41a8dd-fc24-4ebf-8b64-0838fae3fdf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42ae67ee06be4a0ca41cc6442d837d8c)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:22.970317 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41a8dd-fc24-4ebf-8b64-0838fae3fdf6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:23.158918 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:23.163726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a41a8dd-fc24-4ebf-8b64-0838fae3fdf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42ae67ee06be4a0ca41cc6442d837d8c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:23.164384 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a41a8dd-fc24-4ebf-8b64-0838fae3fdf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42ae67ee06be4a0ca41cc6442d837d8c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:23.177080 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a41a8dd-fc24-4ebf-8b64-0838fae3fdf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42ae67ee06be4a0ca41cc6442d837d8c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:23.177801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a41a8dd-fc24-4ebf-8b64-0838fae3fdf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42ae67ee06be4a0ca41cc6442d837d8c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:23.185386 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a41a8dd-fc24-4ebf-8b64-0838fae3fdf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42ae67ee06be4a0ca41cc6442d837d8c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:23.276507 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1120/2205] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:10:22 2022] DELETE /identity/v3/users/42ae67ee06be4a0ca41cc6442d837d8c => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:10:23.281349 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c9b43-4391-430b-a12b-305334378339 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:23.281844 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c9b43-4391-430b-a12b-305334378339 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:23.281958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c9b43-4391-430b-a12b-305334378339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:23.299806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-192c9b43-4391-430b-a12b-305334378339 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:23.301025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-192c9b43-4391-430b-a12b-305334378339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:23.391934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1086/2206] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:23.392863 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41a8dd-fc24-4ebf-8b64-0838fae3fdf6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:23.394300 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0a41a8dd-fc24-4ebf-8b64-0838fae3fdf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:23.464415 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41a8dd-fc24-4ebf-8b64-0838fae3fdf6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:23.468468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea2f0a-3e3e-44d0-8108-0e0962698020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:23.469142 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea2f0a-3e3e-44d0-8108-0e0962698020 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:23.469431 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea2f0a-3e3e-44d0-8108-0e0962698020 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:23.469707 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea2f0a-3e3e-44d0-8108-0e0962698020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:23.479779 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea2f0a-3e3e-44d0-8108-0e0962698020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:23.481756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea2f0a-3e3e-44d0-8108-0e0962698020 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:23.491517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1121/2207] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:10:24.023136 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c9b43-4391-430b-a12b-305334378339 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:24.024924 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-192c9b43-4391-430b-a12b-305334378339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:24.039083 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c9b43-4391-430b-a12b-305334378339 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:24.056579 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48307326-a8ed-43b9-a47d-551bf21205a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:24.057493 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48307326-a8ed-43b9-a47d-551bf21205a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:24.057842 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48307326-a8ed-43b9-a47d-551bf21205a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:24.058044 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48307326-a8ed-43b9-a47d-551bf21205a7 admin admin] PATH_INFO: `/v3/projects/4d4ec30922c44074bcd90beea3ca7f59` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:24.060021 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48307326-a8ed-43b9-a47d-551bf21205a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d4ec30922c44074bcd90beea3ca7f59)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:24.061950 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48307326-a8ed-43b9-a47d-551bf21205a7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:24.190120 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-48307326-a8ed-43b9-a47d-551bf21205a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d4ec30922c44074bcd90beea3ca7f59', 'request_id': 'req-48307326-a8ed-43b9-a47d-551bf21205a7'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:24.275483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-48307326-a8ed-43b9-a47d-551bf21205a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d4ec30922c44074bcd90beea3ca7f59', 'request_id': 'req-48307326-a8ed-43b9-a47d-551bf21205a7'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:24.281536 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-48307326-a8ed-43b9-a47d-551bf21205a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d4ec30922c44074bcd90beea3ca7f59', 'request_id': 'req-48307326-a8ed-43b9-a47d-551bf21205a7'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:24.293073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-48307326-a8ed-43b9-a47d-551bf21205a7 admin admin] The token cache is being invalidate because project 4d4ec30922c44074bcd90beea3ca7f59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:10:24.296180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-48307326-a8ed-43b9-a47d-551bf21205a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:24.296180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-48307326-a8ed-43b9-a47d-551bf21205a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:24.296180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1087/2208] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:10:24 2022] DELETE /identity/v3/projects/4d4ec30922c44074bcd90beea3ca7f59 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:24.300172 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea2f0a-3e3e-44d0-8108-0e0962698020 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:24.310394 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea2f0a-3e3e-44d0-8108-0e0962698020 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:24.369783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64c732-d29e-4c0d-bba7-17ec9a74c1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:24.370772 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64c732-d29e-4c0d-bba7-17ec9a74c1d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:24.370998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64c732-d29e-4c0d-bba7-17ec9a74c1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:24.371310 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64c732-d29e-4c0d-bba7-17ec9a74c1d0 admin admin] PATH_INFO: `/v3/users/1f5f5359f889475b90fdc29693eb4af4` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:24.373179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64c732-d29e-4c0d-bba7-17ec9a74c1d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f5f5359f889475b90fdc29693eb4af4)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:24.375091 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64c732-d29e-4c0d-bba7-17ec9a74c1d0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:24.529126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7d64c732-d29e-4c0d-bba7-17ec9a74c1d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f5f5359f889475b90fdc29693eb4af4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:24.529697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7d64c732-d29e-4c0d-bba7-17ec9a74c1d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f5f5359f889475b90fdc29693eb4af4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:24.541304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:24.542500 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7d64c732-d29e-4c0d-bba7-17ec9a74c1d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f5f5359f889475b90fdc29693eb4af4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:24.543229 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7d64c732-d29e-4c0d-bba7-17ec9a74c1d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f5f5359f889475b90fdc29693eb4af4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:24.550789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7d64c732-d29e-4c0d-bba7-17ec9a74c1d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f5f5359f889475b90fdc29693eb4af4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:24.612818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1122/2209] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:10:24 2022] DELETE /identity/v3/users/1f5f5359f889475b90fdc29693eb4af4 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:24.728553 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48307326-a8ed-43b9-a47d-551bf21205a7 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:24.730670 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-48307326-a8ed-43b9-a47d-551bf21205a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:24.746595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48307326-a8ed-43b9-a47d-551bf21205a7 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:24.783780 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:24.823864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ff690-8163-42cf-b5eb-0b9057168975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:24.824779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ff690-8163-42cf-b5eb-0b9057168975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:24.825070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ff690-8163-42cf-b5eb-0b9057168975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:24.825351 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ff690-8163-42cf-b5eb-0b9057168975 admin admin] PATH_INFO: `/v3/users/98ea4d7dc953432ba9040b6d4b35f107` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:24.826613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ff690-8163-42cf-b5eb-0b9057168975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ea4d7dc953432ba9040b6d4b35f107)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:24.828738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ff690-8163-42cf-b5eb-0b9057168975 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:25.021217 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-642ff690-8163-42cf-b5eb-0b9057168975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ea4d7dc953432ba9040b6d4b35f107'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:25.028406 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-642ff690-8163-42cf-b5eb-0b9057168975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ea4d7dc953432ba9040b6d4b35f107'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:25.029751 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-642ff690-8163-42cf-b5eb-0b9057168975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ea4d7dc953432ba9040b6d4b35f107'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:25.030356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-642ff690-8163-42cf-b5eb-0b9057168975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ea4d7dc953432ba9040b6d4b35f107'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:25.042614 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64c732-d29e-4c0d-bba7-17ec9a74c1d0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:25.044208 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7d64c732-d29e-4c0d-bba7-17ec9a74c1d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:25.045542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-642ff690-8163-42cf-b5eb-0b9057168975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ea4d7dc953432ba9040b6d4b35f107'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:25.057140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64c732-d29e-4c0d-bba7-17ec9a74c1d0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:25.061921 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68c8e1-45c0-4749-a3fb-3f0a7bbe5872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:25.062737 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68c8e1-45c0-4749-a3fb-3f0a7bbe5872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:25.063024 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68c8e1-45c0-4749-a3fb-3f0a7bbe5872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:25.063337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68c8e1-45c0-4749-a3fb-3f0a7bbe5872 admin admin] PATH_INFO: `/v3/projects/38f3e7b79c9c494b981aff1ca39e242a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:25.064533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68c8e1-45c0-4749-a3fb-3f0a7bbe5872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38f3e7b79c9c494b981aff1ca39e242a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:25.066460 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68c8e1-45c0-4749-a3fb-3f0a7bbe5872 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:25.121638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1088/2210] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:10:24 2022] DELETE /identity/v3/users/98ea4d7dc953432ba9040b6d4b35f107 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:25.126138 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e6467-520a-444b-902b-7949dc1921b4 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:25.126440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e6467-520a-444b-902b-7949dc1921b4 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:25.126772 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e6467-520a-444b-902b-7949dc1921b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:25.145118 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-255e6467-520a-444b-902b-7949dc1921b4 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:25.146356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-255e6467-520a-444b-902b-7949dc1921b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:25.171867 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ba68c8e1-45c0-4749-a3fb-3f0a7bbe5872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38f3e7b79c9c494b981aff1ca39e242a', 'request_id': 'req-ba68c8e1-45c0-4749-a3fb-3f0a7bbe5872'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:25.210602 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1089/2211] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:25.218322 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ba68c8e1-45c0-4749-a3fb-3f0a7bbe5872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38f3e7b79c9c494b981aff1ca39e242a', 'request_id': 'req-ba68c8e1-45c0-4749-a3fb-3f0a7bbe5872'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:25.221600 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e6467-520a-444b-902b-7949dc1921b4 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:25.223001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-255e6467-520a-444b-902b-7949dc1921b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:25.227001 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ba68c8e1-45c0-4749-a3fb-3f0a7bbe5872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38f3e7b79c9c494b981aff1ca39e242a', 'request_id': 'req-ba68c8e1-45c0-4749-a3fb-3f0a7bbe5872'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:25.240573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ba68c8e1-45c0-4749-a3fb-3f0a7bbe5872 admin admin] The token cache is being invalidate because project 38f3e7b79c9c494b981aff1ca39e242a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:10:25.240995 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ba68c8e1-45c0-4749-a3fb-3f0a7bbe5872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:25.241781 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ba68c8e1-45c0-4749-a3fb-3f0a7bbe5872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:25.243808 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1123/2212] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:10:25 2022] DELETE /identity/v3/projects/38f3e7b79c9c494b981aff1ca39e242a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:10:25.253317 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc495bb-42de-4a6a-a9bb-6215d3b379ad None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:25.253555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc495bb-42de-4a6a-a9bb-6215d3b379ad None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:25.253869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc495bb-42de-4a6a-a9bb-6215d3b379ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:25.276607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5fc495bb-42de-4a6a-a9bb-6215d3b379ad None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:25.278104 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5fc495bb-42de-4a6a-a9bb-6215d3b379ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:25.291855 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e6467-520a-444b-902b-7949dc1921b4 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:25.297291 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:25.347308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1124/2213] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:25.350857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44428148-b5a9-41d0-a75f-76a961ca25ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:25.351548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44428148-b5a9-41d0-a75f-76a961ca25ed service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:25.351837 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44428148-b5a9-41d0-a75f-76a961ca25ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:25.352033 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abd681-6aa2-49e6-989e-e118da6d6f20 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:25.352181 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44428148-b5a9-41d0-a75f-76a961ca25ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:25.352396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abd681-6aa2-49e6-989e-e118da6d6f20 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:25.352688 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abd681-6aa2-49e6-989e-e118da6d6f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:25.354226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-44428148-b5a9-41d0-a75f-76a961ca25ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:25.368864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44428148-b5a9-41d0-a75f-76a961ca25ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:25.370400 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44428148-b5a9-41d0-a75f-76a961ca25ed service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:25.374080 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-62abd681-6aa2-49e6-989e-e118da6d6f20 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:25.376689 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-62abd681-6aa2-49e6-989e-e118da6d6f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:25.382943 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1090/2214] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:10:25.385450 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1125/2215] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:25.390098 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44428148-b5a9-41d0-a75f-76a961ca25ed service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:25.401884 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44428148-b5a9-41d0-a75f-76a961ca25ed service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:25.405888 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e592f8e-33a8-47cf-8e82-22935d199da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:25.406544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e592f8e-33a8-47cf-8e82-22935d199da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:25.406828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e592f8e-33a8-47cf-8e82-22935d199da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:25.407111 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e592f8e-33a8-47cf-8e82-22935d199da7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:25.407904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e592f8e-33a8-47cf-8e82-22935d199da7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:10:25.408217 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e592f8e-33a8-47cf-8e82-22935d199da7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:25.409833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e592f8e-33a8-47cf-8e82-22935d199da7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:25.422782 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1091/2216] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 19:10:25.427039 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abd681-6aa2-49e6-989e-e118da6d6f20 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:25.437693 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abd681-6aa2-49e6-989e-e118da6d6f20 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:25.448459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ae40c-bb7e-4322-9b2f-5c6b38d44ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:25.449374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ae40c-bb7e-4322-9b2f-5c6b38d44ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:25.449660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ae40c-bb7e-4322-9b2f-5c6b38d44ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:25.449935 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ae40c-bb7e-4322-9b2f-5c6b38d44ee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:25.451285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ae40c-bb7e-4322-9b2f-5c6b38d44ee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:25.453396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ae40c-bb7e-4322-9b2f-5c6b38d44ee4 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:25.524969 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1126/2217] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:10:25 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:10:25.530000 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e592f8e-33a8-47cf-8e82-22935d199da7 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:25.541240 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e592f8e-33a8-47cf-8e82-22935d199da7 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:25.621330 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b876ae8-f1d2-4613-9ff8-a0eaec6675d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:25.621330 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b876ae8-f1d2-4613-9ff8-a0eaec6675d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:25.621330 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b876ae8-f1d2-4613-9ff8-a0eaec6675d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:25.621330 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b876ae8-f1d2-4613-9ff8-a0eaec6675d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:25.621672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b876ae8-f1d2-4613-9ff8-a0eaec6675d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:25.621672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b876ae8-f1d2-4613-9ff8-a0eaec6675d1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:25.709684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1092/2218] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:10:25 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:10:25.710698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ae40c-bb7e-4322-9b2f-5c6b38d44ee4 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:25.721761 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ae40c-bb7e-4322-9b2f-5c6b38d44ee4 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:25.726643 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a206fd-7378-4ec5-a92a-ab5e83ad45c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:25.729870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a206fd-7378-4ec5-a92a-ab5e83ad45c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:25.729870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a206fd-7378-4ec5-a92a-ab5e83ad45c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:25.729870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a206fd-7378-4ec5-a92a-ab5e83ad45c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:25.729870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a206fd-7378-4ec5-a92a-ab5e83ad45c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:25.730283 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a206fd-7378-4ec5-a92a-ab5e83ad45c9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:25.741743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1127/2219] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:10:25.745760 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b876ae8-f1d2-4613-9ff8-a0eaec6675d1 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:25.756697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b876ae8-f1d2-4613-9ff8-a0eaec6675d1 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:25.761773 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1775177-ad21-4c93-8d9c-f46803a827e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:25.762441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1775177-ad21-4c93-8d9c-f46803a827e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:25.762731 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1775177-ad21-4c93-8d9c-f46803a827e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:25.763031 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1775177-ad21-4c93-8d9c-f46803a827e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:25.763790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1775177-ad21-4c93-8d9c-f46803a827e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:25.765493 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1775177-ad21-4c93-8d9c-f46803a827e3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:25.776675 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1093/2220] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:10:25.780576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a206fd-7378-4ec5-a92a-ab5e83ad45c9 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:25.791729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1775177-ad21-4c93-8d9c-f46803a827e3 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:25.795037 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a206fd-7378-4ec5-a92a-ab5e83ad45c9 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:25.795922 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f1775177-ad21-4c93-8d9c-f46803a827e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:25.801304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5c73e-a1d5-49b5-9c64-9aabf551e333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:25.802283 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5c73e-a1d5-49b5-9c64-9aabf551e333 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:25.802571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5c73e-a1d5-49b5-9c64-9aabf551e333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:25.802846 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5c73e-a1d5-49b5-9c64-9aabf551e333 admin admin] PATH_INFO: `/v3/projects/fda97708bb9c497bbd8558563b88d253/users/a3c68d205c364ba2ae49b470c7228985/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:25.818236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1775177-ad21-4c93-8d9c-f46803a827e3 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:25.826693 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5c73e-a1d5-49b5-9c64-9aabf551e333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fda97708bb9c497bbd8558563b88d253, user_id=a3c68d205c364ba2ae49b470c7228985, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:25.827095 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383be8d9-9c8c-44cd-b58e-ff32e8f6c4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:25.827095 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383be8d9-9c8c-44cd-b58e-ff32e8f6c4c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:25.827305 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5c73e-a1d5-49b5-9c64-9aabf551e333 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:25.827384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383be8d9-9c8c-44cd-b58e-ff32e8f6c4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:25.827384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383be8d9-9c8c-44cd-b58e-ff32e8f6c4c5 admin admin] PATH_INFO: `/v3/projects/66ff5513df764ccbb073b2aaf8fb3336` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:25.829159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383be8d9-9c8c-44cd-b58e-ff32e8f6c4c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66ff5513df764ccbb073b2aaf8fb3336)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:25.831280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383be8d9-9c8c-44cd-b58e-ff32e8f6c4c5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:25.905501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:25.910540 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1128/2221] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:10:25 2022] PUT /identity/v3/projects/fda97708bb9c497bbd8558563b88d253/users/a3c68d205c364ba2ae49b470c7228985/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:25.921672 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a99af8-7558-41ae-a439-59994b59bb65 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:25.922076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a99af8-7558-41ae-a439-59994b59bb65 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:25.922411 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a99af8-7558-41ae-a439-59994b59bb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:25.951606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-08a99af8-7558-41ae-a439-59994b59bb65 None None] MFA Rules not processed for user `a3c68d205c364ba2ae49b470c7228985`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:25.953063 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-08a99af8-7558-41ae-a439-59994b59bb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:25.973484 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-383be8d9-9c8c-44cd-b58e-ff32e8f6c4c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66ff5513df764ccbb073b2aaf8fb3336', 'request_id': 'req-383be8d9-9c8c-44cd-b58e-ff32e8f6c4c5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:26.040933 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1129/2222] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:26.052899 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a99af8-7558-41ae-a439-59994b59bb65 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:26.054328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-08a99af8-7558-41ae-a439-59994b59bb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:26.080441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-383be8d9-9c8c-44cd-b58e-ff32e8f6c4c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66ff5513df764ccbb073b2aaf8fb3336', 'request_id': 'req-383be8d9-9c8c-44cd-b58e-ff32e8f6c4c5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:26.088949 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-383be8d9-9c8c-44cd-b58e-ff32e8f6c4c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66ff5513df764ccbb073b2aaf8fb3336', 'request_id': 'req-383be8d9-9c8c-44cd-b58e-ff32e8f6c4c5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:26.099577 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:26.103765 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-383be8d9-9c8c-44cd-b58e-ff32e8f6c4c5 admin admin] The token cache is being invalidate because project 66ff5513df764ccbb073b2aaf8fb3336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:10:26.104060 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-383be8d9-9c8c-44cd-b58e-ff32e8f6c4c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:26.104750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-383be8d9-9c8c-44cd-b58e-ff32e8f6c4c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:26.106397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1094/2223] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:10:25 2022] DELETE /identity/v3/projects/66ff5513df764ccbb073b2aaf8fb3336 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:26.111012 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383be8d9-9c8c-44cd-b58e-ff32e8f6c4c5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:26.112646 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-383be8d9-9c8c-44cd-b58e-ff32e8f6c4c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:26.120903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a99af8-7558-41ae-a439-59994b59bb65 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:26.181512 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4438cc1-9d44-47fd-a05e-b92d4783d872 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:26.182254 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4438cc1-9d44-47fd-a05e-b92d4783d872 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:26.182544 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4438cc1-9d44-47fd-a05e-b92d4783d872 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:26.182821 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4438cc1-9d44-47fd-a05e-b92d4783d872 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:26.184663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b4438cc1-9d44-47fd-a05e-b92d4783d872 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:26.187495 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383be8d9-9c8c-44cd-b58e-ff32e8f6c4c5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:26.192249 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c54029-0503-46ad-9ec6-4f382303059e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:26.193128 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c54029-0503-46ad-9ec6-4f382303059e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:26.193422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c54029-0503-46ad-9ec6-4f382303059e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:26.193703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c54029-0503-46ad-9ec6-4f382303059e admin admin] PATH_INFO: `/v3/users/228983ffa26f42449160eb76b1dce79d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:26.194919 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c54029-0503-46ad-9ec6-4f382303059e admin admin] RBAC: Authorizing `identity:delete_user(user_id=228983ffa26f42449160eb76b1dce79d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:26.196932 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c54029-0503-46ad-9ec6-4f382303059e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:26.253700 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4438cc1-9d44-47fd-a05e-b92d4783d872 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:26.255315 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4438cc1-9d44-47fd-a05e-b92d4783d872 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:26.259990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:26.264858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1130/2224] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:10:26.285705 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4438cc1-9d44-47fd-a05e-b92d4783d872 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:26.286776 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b4438cc1-9d44-47fd-a05e-b92d4783d872 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:26.304167 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4438cc1-9d44-47fd-a05e-b92d4783d872 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:26.305910 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c324723-2b70-49d0-8115-27c4617675a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:26.306792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c324723-2b70-49d0-8115-27c4617675a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:26.307082 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c324723-2b70-49d0-8115-27c4617675a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:26.307400 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c324723-2b70-49d0-8115-27c4617675a9 admin admin] PATH_INFO: `/v3/users/a3c68d205c364ba2ae49b470c7228985` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:26.308609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c324723-2b70-49d0-8115-27c4617675a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3c68d205c364ba2ae49b470c7228985)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:26.310550 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c324723-2b70-49d0-8115-27c4617675a9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:26.346299 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-18c54029-0503-46ad-9ec6-4f382303059e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '228983ffa26f42449160eb76b1dce79d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:26.353349 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-18c54029-0503-46ad-9ec6-4f382303059e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '228983ffa26f42449160eb76b1dce79d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:26.354068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-18c54029-0503-46ad-9ec6-4f382303059e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '228983ffa26f42449160eb76b1dce79d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:26.354699 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-18c54029-0503-46ad-9ec6-4f382303059e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '228983ffa26f42449160eb76b1dce79d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:26.367030 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-18c54029-0503-46ad-9ec6-4f382303059e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '228983ffa26f42449160eb76b1dce79d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:26.474827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1095/2225] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:10:26 2022] DELETE /identity/v3/users/228983ffa26f42449160eb76b1dce79d => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:10:26.507420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4c324723-2b70-49d0-8115-27c4617675a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3c68d205c364ba2ae49b470c7228985'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:26.508189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4c324723-2b70-49d0-8115-27c4617675a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3c68d205c364ba2ae49b470c7228985'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:26.521166 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4c324723-2b70-49d0-8115-27c4617675a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3c68d205c364ba2ae49b470c7228985'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:26.521868 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4c324723-2b70-49d0-8115-27c4617675a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3c68d205c364ba2ae49b470c7228985'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:26.529482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4c324723-2b70-49d0-8115-27c4617675a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3c68d205c364ba2ae49b470c7228985'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:26.590801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1131/2226] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:10:26 2022] DELETE /identity/v3/users/a3c68d205c364ba2ae49b470c7228985 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:10:26.596007 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b7def-2cab-45e1-98e9-6af917e7ebe8 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:26.596300 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b7def-2cab-45e1-98e9-6af917e7ebe8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:26.596583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b7def-2cab-45e1-98e9-6af917e7ebe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:26.613992 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-5e4b7def-2cab-45e1-98e9-6af917e7ebe8 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:26.615222 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5e4b7def-2cab-45e1-98e9-6af917e7ebe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:26.680833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1096/2227] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:26.707058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c324723-2b70-49d0-8115-27c4617675a9 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:26.708561 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4c324723-2b70-49d0-8115-27c4617675a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:26.782363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c324723-2b70-49d0-8115-27c4617675a9 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:26.787432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a4644-4ff7-4467-9b1f-6da4a54ce49b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:26.788284 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a4644-4ff7-4467-9b1f-6da4a54ce49b service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:26.789322 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a4644-4ff7-4467-9b1f-6da4a54ce49b service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:26.789781 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a4644-4ff7-4467-9b1f-6da4a54ce49b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:26.802658 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a4644-4ff7-4467-9b1f-6da4a54ce49b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:26.804973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a4644-4ff7-4467-9b1f-6da4a54ce49b service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:26.823602 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1132/2228] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:10:26.954165 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b7def-2cab-45e1-98e9-6af917e7ebe8 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:26.955667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5e4b7def-2cab-45e1-98e9-6af917e7ebe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:26.967995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b7def-2cab-45e1-98e9-6af917e7ebe8 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:26.973237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08af24-ce1a-45c4-b0a4-9026a28fe9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:26.974126 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08af24-ce1a-45c4-b0a4-9026a28fe9d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:26.974588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08af24-ce1a-45c4-b0a4-9026a28fe9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:26.975425 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08af24-ce1a-45c4-b0a4-9026a28fe9d7 admin admin] PATH_INFO: `/v3/projects/62a13131a37f4d5f955b55889e12c2d7` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:26.976251 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08af24-ce1a-45c4-b0a4-9026a28fe9d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62a13131a37f4d5f955b55889e12c2d7)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:26.978170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08af24-ce1a-45c4-b0a4-9026a28fe9d7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:27.003540 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:27.099308 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7d08af24-ce1a-45c4-b0a4-9026a28fe9d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62a13131a37f4d5f955b55889e12c2d7', 'request_id': 'req-7d08af24-ce1a-45c4-b0a4-9026a28fe9d7'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:27.155762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7d08af24-ce1a-45c4-b0a4-9026a28fe9d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62a13131a37f4d5f955b55889e12c2d7', 'request_id': 'req-7d08af24-ce1a-45c4-b0a4-9026a28fe9d7'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:27.161947 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7d08af24-ce1a-45c4-b0a4-9026a28fe9d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62a13131a37f4d5f955b55889e12c2d7', 'request_id': 'req-7d08af24-ce1a-45c4-b0a4-9026a28fe9d7'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:27.174047 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7d08af24-ce1a-45c4-b0a4-9026a28fe9d7 admin admin] The token cache is being invalidate because project 62a13131a37f4d5f955b55889e12c2d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:10:27.174339 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7d08af24-ce1a-45c4-b0a4-9026a28fe9d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:27.174989 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7d08af24-ce1a-45c4-b0a4-9026a28fe9d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:27.176588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1097/2229] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:10:26 2022] DELETE /identity/v3/projects/62a13131a37f4d5f955b55889e12c2d7 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:27.181823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364caf0-8f80-4e3c-a020-837db3bd0075 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:27.182156 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364caf0-8f80-4e3c-a020-837db3bd0075 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:27.182451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364caf0-8f80-4e3c-a020-837db3bd0075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:27.201695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-a364caf0-8f80-4e3c-a020-837db3bd0075 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:27.203008 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a364caf0-8f80-4e3c-a020-837db3bd0075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:27.265252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1133/2230] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:27.273192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a5380-ad6f-43be-9b74-cd99d25e4e26 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:27.273539 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a5380-ad6f-43be-9b74-cd99d25e4e26 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:27.275356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a5380-ad6f-43be-9b74-cd99d25e4e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:27.292146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d39a5380-ad6f-43be-9b74-cd99d25e4e26 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:27.293376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d39a5380-ad6f-43be-9b74-cd99d25e4e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:27.302034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1098/2231] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:27.306154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364caf0-8f80-4e3c-a020-837db3bd0075 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:27.317328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364caf0-8f80-4e3c-a020-837db3bd0075 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:27.321952 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f906a-280a-4a2a-a2ed-d18ea41ee1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:27.322606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f906a-280a-4a2a-a2ed-d18ea41ee1be admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:27.322889 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f906a-280a-4a2a-a2ed-d18ea41ee1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:27.323185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f906a-280a-4a2a-a2ed-d18ea41ee1be admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:27.324012 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f906a-280a-4a2a-a2ed-d18ea41ee1be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:10:27.324291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f906a-280a-4a2a-a2ed-d18ea41ee1be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:27.326015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f906a-280a-4a2a-a2ed-d18ea41ee1be admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:27.337867 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1134/2232] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 19:10:27.342215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a5380-ad6f-43be-9b74-cd99d25e4e26 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:27.351746 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a5380-ad6f-43be-9b74-cd99d25e4e26 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:27.356416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f66a1a-da95-401b-80d5-395469148f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:27.357599 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f66a1a-da95-401b-80d5-395469148f22 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:27.357885 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f66a1a-da95-401b-80d5-395469148f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:27.358160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f66a1a-da95-401b-80d5-395469148f22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:27.358879 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f66a1a-da95-401b-80d5-395469148f22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:27.360895 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f66a1a-da95-401b-80d5-395469148f22 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:27.368552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f906a-280a-4a2a-a2ed-d18ea41ee1be admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:27.370438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c17f906a-280a-4a2a-a2ed-d18ea41ee1be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:27.380727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:27.387008 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f906a-280a-4a2a-a2ed-d18ea41ee1be admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:27.391909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa522f-43b3-4982-9ea1-17df178e8e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:27.392794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa522f-43b3-4982-9ea1-17df178e8e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:27.393152 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa522f-43b3-4982-9ea1-17df178e8e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:27.393459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa522f-43b3-4982-9ea1-17df178e8e8f admin admin] PATH_INFO: `/v3/projects/fda97708bb9c497bbd8558563b88d253` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:27.394672 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa522f-43b3-4982-9ea1-17df178e8e8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fda97708bb9c497bbd8558563b88d253)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:27.396642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa522f-43b3-4982-9ea1-17df178e8e8f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:27.448473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1099/2233] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:10:27 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:10:27.452243 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f66a1a-da95-401b-80d5-395469148f22 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:27.493504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fcfa522f-43b3-4982-9ea1-17df178e8e8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fda97708bb9c497bbd8558563b88d253', 'request_id': 'req-fcfa522f-43b3-4982-9ea1-17df178e8e8f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:27.515827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f66a1a-da95-401b-80d5-395469148f22 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:27.537513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fcfa522f-43b3-4982-9ea1-17df178e8e8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fda97708bb9c497bbd8558563b88d253', 'request_id': 'req-fcfa522f-43b3-4982-9ea1-17df178e8e8f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:27.543523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fcfa522f-43b3-4982-9ea1-17df178e8e8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fda97708bb9c497bbd8558563b88d253', 'request_id': 'req-fcfa522f-43b3-4982-9ea1-17df178e8e8f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:27.556963 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fcfa522f-43b3-4982-9ea1-17df178e8e8f admin admin] The token cache is being invalidate because project fda97708bb9c497bbd8558563b88d253 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:10:27.557255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fcfa522f-43b3-4982-9ea1-17df178e8e8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:27.558150 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fcfa522f-43b3-4982-9ea1-17df178e8e8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:27.560420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1135/2234] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:10:27 2022] DELETE /identity/v3/projects/fda97708bb9c497bbd8558563b88d253 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:10:27.565067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c826d0b-4162-4418-b717-9ad6b2ecffe8 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:27.565423 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c826d0b-4162-4418-b717-9ad6b2ecffe8 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:27.565737 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c826d0b-4162-4418-b717-9ad6b2ecffe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:27.572108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d6fbe-0562-4be3-8e05-5d80e14653ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:27.572784 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d6fbe-0562-4be3-8e05-5d80e14653ac admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:27.573073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d6fbe-0562-4be3-8e05-5d80e14653ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:27.573358 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d6fbe-0562-4be3-8e05-5d80e14653ac admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:27.574111 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d6fbe-0562-4be3-8e05-5d80e14653ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:27.576267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d6fbe-0562-4be3-8e05-5d80e14653ac admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:27.584065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5c826d0b-4162-4418-b717-9ad6b2ecffe8 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:27.585273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5c826d0b-4162-4418-b717-9ad6b2ecffe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:27.593789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1136/2235] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:27.602324 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d6752-6be1-4b01-b0ce-ef2c0974a2b4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:27.602543 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d6752-6be1-4b01-b0ce-ef2c0974a2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:27.602822 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d6752-6be1-4b01-b0ce-ef2c0974a2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:27.621806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-473d6752-6be1-4b01-b0ce-ef2c0974a2b4 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:27.625046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-473d6752-6be1-4b01-b0ce-ef2c0974a2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:27.631021 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1137/2236] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:27.635032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d6752-6be1-4b01-b0ce-ef2c0974a2b4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:27.643822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1100/2237] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:10:27 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:10:27.644856 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d6fbe-0562-4be3-8e05-5d80e14653ac admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:27.646536 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6f7d6fbe-0562-4be3-8e05-5d80e14653ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:27.646671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d6752-6be1-4b01-b0ce-ef2c0974a2b4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:27.652606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97b1d2-a622-4718-a2db-dbb98636d730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:27.653270 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97b1d2-a622-4718-a2db-dbb98636d730 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:27.654093 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97b1d2-a622-4718-a2db-dbb98636d730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:27.654372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97b1d2-a622-4718-a2db-dbb98636d730 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:27.655100 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97b1d2-a622-4718-a2db-dbb98636d730 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:10:27.655455 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97b1d2-a622-4718-a2db-dbb98636d730 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:27.657133 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97b1d2-a622-4718-a2db-dbb98636d730 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:27.659211 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d6fbe-0562-4be3-8e05-5d80e14653ac admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:27.664692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e33e2-09ef-4e5d-ab16-d1f55ceccd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:27.665393 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e33e2-09ef-4e5d-ab16-d1f55ceccd5e admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:27.665688 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e33e2-09ef-4e5d-ab16-d1f55ceccd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:27.666421 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e33e2-09ef-4e5d-ab16-d1f55ceccd5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:27.667177 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e33e2-09ef-4e5d-ab16-d1f55ceccd5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:27.668806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e33e2-09ef-4e5d-ab16-d1f55ceccd5e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:27.669930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1138/2238] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 19:10:27.674131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97b1d2-a622-4718-a2db-dbb98636d730 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:27.675633 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:27.680631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1101/2239] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:10:27.684055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97b1d2-a622-4718-a2db-dbb98636d730 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:27.688873 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbd171-7847-44d1-bde9-76eb348bb736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:27.689578 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbd171-7847-44d1-bde9-76eb348bb736 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:27.689871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbd171-7847-44d1-bde9-76eb348bb736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:27.690153 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbd171-7847-44d1-bde9-76eb348bb736 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:27.690853 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dbd171-7847-44d1-bde9-76eb348bb736 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:27.691872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e33e2-09ef-4e5d-ab16-d1f55ceccd5e admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:27.693054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dbd171-7847-44d1-bde9-76eb348bb736 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:27.702552 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e33e2-09ef-4e5d-ab16-d1f55ceccd5e admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:27.708141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76559a4-810d-4122-8785-ba0ceca9a718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:27.708806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76559a4-810d-4122-8785-ba0ceca9a718 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:27.709093 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76559a4-810d-4122-8785-ba0ceca9a718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:27.709372 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76559a4-810d-4122-8785-ba0ceca9a718 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:27.710103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76559a4-810d-4122-8785-ba0ceca9a718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:27.711736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76559a4-810d-4122-8785-ba0ceca9a718 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:27.723135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1102/2240] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:10:27.727308 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76559a4-810d-4122-8785-ba0ceca9a718 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:27.737771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76559a4-810d-4122-8785-ba0ceca9a718 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:27.743077 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73298f26-a8dd-4159-94e4-5eeb1b3d3440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:27.743273 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73298f26-a8dd-4159-94e4-5eeb1b3d3440 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:27.743478 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73298f26-a8dd-4159-94e4-5eeb1b3d3440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:27.743944 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73298f26-a8dd-4159-94e4-5eeb1b3d3440 admin admin] PATH_INFO: `/v3/projects/233dcf4525d540b581f144acee94c18d/users/54e3bdd8f01b4ee5be6ac5e6bc44549b/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:27.758075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73298f26-a8dd-4159-94e4-5eeb1b3d3440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=233dcf4525d540b581f144acee94c18d, user_id=54e3bdd8f01b4ee5be6ac5e6bc44549b, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:27.760744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73298f26-a8dd-4159-94e4-5eeb1b3d3440 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:27.767698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1139/2241] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:10:27 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:10:27.771895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbd171-7847-44d1-bde9-76eb348bb736 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:27.834220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbd171-7847-44d1-bde9-76eb348bb736 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:27.838937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af1d97-a073-4042-8497-95321e897340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:27.839683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af1d97-a073-4042-8497-95321e897340 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:27.839950 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af1d97-a073-4042-8497-95321e897340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:27.840226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af1d97-a073-4042-8497-95321e897340 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:27.840917 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af1d97-a073-4042-8497-95321e897340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:27.843435 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af1d97-a073-4042-8497-95321e897340 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:27.855256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1103/2242] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:10:27 2022] PUT /identity/v3/projects/233dcf4525d540b581f144acee94c18d/users/54e3bdd8f01b4ee5be6ac5e6bc44549b/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:10:27.857991 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99c22c-f5b2-400e-8b92-6210098e9126 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:27.858292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99c22c-f5b2-400e-8b92-6210098e9126 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:27.858575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99c22c-f5b2-400e-8b92-6210098e9126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:27.875477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4f99c22c-f5b2-400e-8b92-6210098e9126 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:27.876693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4f99c22c-f5b2-400e-8b92-6210098e9126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:27.946406 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1104/2243] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:27.950139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1140/2244] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:10:27 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:10:27.952606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af1d97-a073-4042-8497-95321e897340 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:27.963897 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af1d97-a073-4042-8497-95321e897340 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:27.969403 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc95049-c981-423b-b2cb-ccfadfc5ff43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:27.970082 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc95049-c981-423b-b2cb-ccfadfc5ff43 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:27.970408 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc95049-c981-423b-b2cb-ccfadfc5ff43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:27.970694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc95049-c981-423b-b2cb-ccfadfc5ff43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:27.971464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc95049-c981-423b-b2cb-ccfadfc5ff43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:27.973091 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc95049-c981-423b-b2cb-ccfadfc5ff43 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:27.974937 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99c22c-f5b2-400e-8b92-6210098e9126 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:27.976651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4f99c22c-f5b2-400e-8b92-6210098e9126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:27.988242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1141/2245] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:10:27.992738 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc95049-c981-423b-b2cb-ccfadfc5ff43 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:28.011000 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc95049-c981-423b-b2cb-ccfadfc5ff43 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:28.018476 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ab863-a800-4013-9e58-906999692d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:28.019197 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ab863-a800-4013-9e58-906999692d17 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:28.019503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ab863-a800-4013-9e58-906999692d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:28.019787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ab863-a800-4013-9e58-906999692d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:28.020561 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ab863-a800-4013-9e58-906999692d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:28.023079 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ab863-a800-4013-9e58-906999692d17 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:28.039476 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1142/2246] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:10:28.040592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ab863-a800-4013-9e58-906999692d17 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:28.046993 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:28.052422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ab863-a800-4013-9e58-906999692d17 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:28.058715 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abcaec-cf7c-4ca6-81b2-9c0144f765a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:28.059954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abcaec-cf7c-4ca6-81b2-9c0144f765a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:28.060589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abcaec-cf7c-4ca6-81b2-9c0144f765a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:28.060589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abcaec-cf7c-4ca6-81b2-9c0144f765a6 admin admin] PATH_INFO: `/v3/projects/45825d43d2ac4b349b7dbee42a0ebc47/users/49aca23c1ad24971bd54a411c6c2da18/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:28.073140 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99c22c-f5b2-400e-8b92-6210098e9126 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:28.077687 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9eedc1-9dcf-40c7-b126-93ec264fd68b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:28.078062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81abcaec-cf7c-4ca6-81b2-9c0144f765a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45825d43d2ac4b349b7dbee42a0ebc47, user_id=49aca23c1ad24971bd54a411c6c2da18, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:28.078580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9eedc1-9dcf-40c7-b126-93ec264fd68b service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:28.079065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9eedc1-9dcf-40c7-b126-93ec264fd68b service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:28.079385 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9eedc1-9dcf-40c7-b126-93ec264fd68b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:28.081062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81abcaec-cf7c-4ca6-81b2-9c0144f765a6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:28.089544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9eedc1-9dcf-40c7-b126-93ec264fd68b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:28.091574 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9eedc1-9dcf-40c7-b126-93ec264fd68b service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:28.101639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1105/2247] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:10:28.144423 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1143/2248] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:10:28 2022] PUT /identity/v3/projects/45825d43d2ac4b349b7dbee42a0ebc47/users/49aca23c1ad24971bd54a411c6c2da18/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:28.155608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06843b52-28cc-46a3-b6fa-d76193e898b9 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:28.155921 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06843b52-28cc-46a3-b6fa-d76193e898b9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:28.156251 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06843b52-28cc-46a3-b6fa-d76193e898b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:28.183019 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-06843b52-28cc-46a3-b6fa-d76193e898b9 None None] MFA Rules not processed for user `49aca23c1ad24971bd54a411c6c2da18`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:28.184293 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-06843b52-28cc-46a3-b6fa-d76193e898b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:28.246576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:28.252457 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1106/2249] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:28.257302 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abcaec-cf7c-4ca6-81b2-9c0144f765a6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:28.324086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abcaec-cf7c-4ca6-81b2-9c0144f765a6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:28.329582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cd7dc-ea35-48ea-9743-8cab413c7ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:28.330252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cd7dc-ea35-48ea-9743-8cab413c7ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:28.330542 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cd7dc-ea35-48ea-9743-8cab413c7ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:28.331140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cd7dc-ea35-48ea-9743-8cab413c7ca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:28.331718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77cd7dc-ea35-48ea-9743-8cab413c7ca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:28.333662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77cd7dc-ea35-48ea-9743-8cab413c7ca4 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:28.397450 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1144/2250] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:10:28 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:10:28.401914 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06843b52-28cc-46a3-b6fa-d76193e898b9 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:28.413074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06843b52-28cc-46a3-b6fa-d76193e898b9 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:28.473882 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71ae23-10e1-4d49-a050-7310a141d679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:28.474629 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71ae23-10e1-4d49-a050-7310a141d679 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:28.474919 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71ae23-10e1-4d49-a050-7310a141d679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:28.475287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71ae23-10e1-4d49-a050-7310a141d679 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:28.475987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71ae23-10e1-4d49-a050-7310a141d679 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:28.477945 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71ae23-10e1-4d49-a050-7310a141d679 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:28.532555 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1107/2251] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:10:28 2022] POST /identity/v3/users => generated 289 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:10:28.536660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cd7dc-ea35-48ea-9743-8cab413c7ca4 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:28.547770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cd7dc-ea35-48ea-9743-8cab413c7ca4 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:28.552728 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20633596-b9a7-4a85-ba05-bb4e193f3fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:28.553454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20633596-b9a7-4a85-ba05-bb4e193f3fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:28.553745 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20633596-b9a7-4a85-ba05-bb4e193f3fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:28.554046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20633596-b9a7-4a85-ba05-bb4e193f3fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:28.554762 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20633596-b9a7-4a85-ba05-bb4e193f3fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:28.556422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20633596-b9a7-4a85-ba05-bb4e193f3fa9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:28.571446 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1145/2252] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:10:28.572753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71ae23-10e1-4d49-a050-7310a141d679 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:28.584240 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71ae23-10e1-4d49-a050-7310a141d679 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:28.589011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc76768-7a8e-4cab-a8b0-7251ba7039c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:28.589791 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc76768-7a8e-4cab-a8b0-7251ba7039c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:28.589982 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc76768-7a8e-4cab-a8b0-7251ba7039c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:28.590263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc76768-7a8e-4cab-a8b0-7251ba7039c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:28.591025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc76768-7a8e-4cab-a8b0-7251ba7039c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:28.592658 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc76768-7a8e-4cab-a8b0-7251ba7039c6 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:28.610804 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1108/2253] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:10:28.611905 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20633596-b9a7-4a85-ba05-bb4e193f3fa9 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:28.622834 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20633596-b9a7-4a85-ba05-bb4e193f3fa9 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:28.627690 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385a1c2-62e2-454e-bc5c-eb627e2a0a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:28.628576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385a1c2-62e2-454e-bc5c-eb627e2a0a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:28.628864 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385a1c2-62e2-454e-bc5c-eb627e2a0a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:28.629143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385a1c2-62e2-454e-bc5c-eb627e2a0a19 admin admin] PATH_INFO: `/v3/projects/883e09cb29de4f85a78ae6304a495d12/users/4a46abfb1edf4d0c9c097923f933e09f/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:28.645933 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385a1c2-62e2-454e-bc5c-eb627e2a0a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=883e09cb29de4f85a78ae6304a495d12, user_id=4a46abfb1edf4d0c9c097923f933e09f, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:28.649076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385a1c2-62e2-454e-bc5c-eb627e2a0a19 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:28.710273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1146/2254] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:10:28 2022] PUT /identity/v3/projects/883e09cb29de4f85a78ae6304a495d12/users/4a46abfb1edf4d0c9c097923f933e09f/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:28.721643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bd4cf-4cc0-4289-ade2-5101116a6640 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:28.721937 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bd4cf-4cc0-4289-ade2-5101116a6640 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:28.722223 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bd4cf-4cc0-4289-ade2-5101116a6640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:28.751461 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-981bd4cf-4cc0-4289-ade2-5101116a6640 None None] MFA Rules not processed for user `4a46abfb1edf4d0c9c097923f933e09f`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:28.752742 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-981bd4cf-4cc0-4289-ade2-5101116a6640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:28.820378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1109/2255] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:28.824848 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385a1c2-62e2-454e-bc5c-eb627e2a0a19 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:28.887504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385a1c2-62e2-454e-bc5c-eb627e2a0a19 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:28.892508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a715a84-17eb-4408-bf60-d2404ab5b515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:28.893192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a715a84-17eb-4408-bf60-d2404ab5b515 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:28.895903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a715a84-17eb-4408-bf60-d2404ab5b515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:28.895903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a715a84-17eb-4408-bf60-d2404ab5b515 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:28.895903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a715a84-17eb-4408-bf60-d2404ab5b515 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:28.897256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a715a84-17eb-4408-bf60-d2404ab5b515 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:29.004016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1147/2256] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:10:28 2022] POST /identity/v3/projects => generated 375 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:10:29.008094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bd4cf-4cc0-4289-ade2-5101116a6640 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:29.020319 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bd4cf-4cc0-4289-ade2-5101116a6640 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:29.081214 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52cbf1-13b3-4cc2-be03-e47958bf8cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:29.081892 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52cbf1-13b3-4cc2-be03-e47958bf8cae admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:29.082178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52cbf1-13b3-4cc2-be03-e47958bf8cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:29.082459 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52cbf1-13b3-4cc2-be03-e47958bf8cae admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:29.083159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52cbf1-13b3-4cc2-be03-e47958bf8cae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:29.085339 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52cbf1-13b3-4cc2-be03-e47958bf8cae admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:29.095925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:29.207978 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1110/2257] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:10:29 2022] POST /identity/v3/users => generated 294 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:10:29.211980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a715a84-17eb-4408-bf60-d2404ab5b515 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:29.223864 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a715a84-17eb-4408-bf60-d2404ab5b515 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:29.229929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c57a9-d03c-49cd-a859-d6460389be01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:29.230357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c57a9-d03c-49cd-a859-d6460389be01 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:29.230687 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c57a9-d03c-49cd-a859-d6460389be01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:29.230967 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c57a9-d03c-49cd-a859-d6460389be01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:29.232448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c57a9-d03c-49cd-a859-d6460389be01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:29.234508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c57a9-d03c-49cd-a859-d6460389be01 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:29.246912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1148/2258] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:10:29.251233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52cbf1-13b3-4cc2-be03-e47958bf8cae admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:29.264254 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52cbf1-13b3-4cc2-be03-e47958bf8cae admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:29.269669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0a029-e2b0-44c8-99e7-17bfc1dd3e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:29.269876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0a029-e2b0-44c8-99e7-17bfc1dd3e96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:29.270184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0a029-e2b0-44c8-99e7-17bfc1dd3e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:29.270488 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0a029-e2b0-44c8-99e7-17bfc1dd3e96 admin admin] PATH_INFO: `/v3/projects/b8dfb078c0644c659cf2f2ef4d7a0c46/users/54e280b2bb1545c6806ae9a59b608d30/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:29.287285 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0a029-e2b0-44c8-99e7-17bfc1dd3e96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8dfb078c0644c659cf2f2ef4d7a0c46, user_id=54e280b2bb1545c6806ae9a59b608d30, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:29.290104 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0a029-e2b0-44c8-99e7-17bfc1dd3e96 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:29.328507 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd674e9b-9030-4c67-bb90-22944b4d25e4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:29.328825 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd674e9b-9030-4c67-bb90-22944b4d25e4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:29.329106 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd674e9b-9030-4c67-bb90-22944b4d25e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:29.336085 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:29.357166 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-fd674e9b-9030-4c67-bb90-22944b4d25e4 None None] MFA Rules not processed for user `54e3bdd8f01b4ee5be6ac5e6bc44549b`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:29.358368 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-fd674e9b-9030-4c67-bb90-22944b4d25e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:29.401753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1111/2259] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:10:29 2022] PUT /identity/v3/projects/b8dfb078c0644c659cf2f2ef4d7a0c46/users/54e280b2bb1545c6806ae9a59b608d30/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:29.406122 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0a029-e2b0-44c8-99e7-17bfc1dd3e96 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:29.425942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1149/2260] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:29.438231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd674e9b-9030-4c67-bb90-22944b4d25e4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:29.439728 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-fd674e9b-9030-4c67-bb90-22944b4d25e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:29.473135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0a029-e2b0-44c8-99e7-17bfc1dd3e96 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:29.478049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0da2d-0982-4e05-99e5-c79ecd0e9622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:29.478752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0da2d-0982-4e05-99e5-c79ecd0e9622 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:29.479022 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0da2d-0982-4e05-99e5-c79ecd0e9622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:29.479340 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0da2d-0982-4e05-99e5-c79ecd0e9622 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:29.480072 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0da2d-0982-4e05-99e5-c79ecd0e9622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:29.482252 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0da2d-0982-4e05-99e5-c79ecd0e9622 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:29.492728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1112/2261] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:10:29.497082 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0da2d-0982-4e05-99e5-c79ecd0e9622 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:29.502029 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd674e9b-9030-4c67-bb90-22944b4d25e4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:29.507427 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0da2d-0982-4e05-99e5-c79ecd0e9622 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:29.513730 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114319b1-3437-428c-8dc7-9cc31dfb2f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:29.514617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114319b1-3437-428c-8dc7-9cc31dfb2f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:29.514897 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114319b1-3437-428c-8dc7-9cc31dfb2f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:29.515197 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114319b1-3437-428c-8dc7-9cc31dfb2f74 admin admin] PATH_INFO: `/v3/projects/b8dfb078c0644c659cf2f2ef4d7a0c46/users/54e280b2bb1545c6806ae9a59b608d30/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:29.517384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114319b1-3437-428c-8dc7-9cc31dfb2f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8dfb078c0644c659cf2f2ef4d7a0c46, user_id=54e280b2bb1545c6806ae9a59b608d30, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:29.521748 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114319b1-3437-428c-8dc7-9cc31dfb2f74 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:29.542771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1113/2262] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:10:29 2022] PUT /identity/v3/projects/b8dfb078c0644c659cf2f2ef4d7a0c46/users/54e280b2bb1545c6806ae9a59b608d30/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:29.544453 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531e47c-c851-496f-bd38-087ffefa3931 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:29.544746 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531e47c-c851-496f-bd38-087ffefa3931 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:29.545025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531e47c-c851-496f-bd38-087ffefa3931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:29.556818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dcbcf4-aa27-40eb-98f5-3d307304e82a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:29.557496 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcbcf4-aa27-40eb-98f5-3d307304e82a service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:29.557772 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcbcf4-aa27-40eb-98f5-3d307304e82a service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:29.558049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcbcf4-aa27-40eb-98f5-3d307304e82a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:29.574653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7531e47c-c851-496f-bd38-087ffefa3931 None None] MFA Rules not processed for user `54e280b2bb1545c6806ae9a59b608d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:29.576830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7531e47c-c851-496f-bd38-087ffefa3931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:29.617775 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dcbcf4-aa27-40eb-98f5-3d307304e82a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:29.619256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dcbcf4-aa27-40eb-98f5-3d307304e82a service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:29.628559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1150/2263] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:10:29.661092 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1114/2264] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:29.672743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dcbcf4-aa27-40eb-98f5-3d307304e82a service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:29.674298 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a0dcbcf4-aa27-40eb-98f5-3d307304e82a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:29.682098 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:29.685715 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dcbcf4-aa27-40eb-98f5-3d307304e82a service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:29.736351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a858c-7060-47f0-bed1-cb07f69567c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:29.737007 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a858c-7060-47f0-bed1-cb07f69567c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:29.737296 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a858c-7060-47f0-bed1-cb07f69567c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:29.737578 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a858c-7060-47f0-bed1-cb07f69567c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:29.798948 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a858c-7060-47f0-bed1-cb07f69567c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:29.800474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a858c-7060-47f0-bed1-cb07f69567c2 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:29.809540 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1151/2265] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:10:32.522173 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14ec8d-210e-4b81-9bda-a01fa22e107c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:32.522921 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14ec8d-210e-4b81-9bda-a01fa22e107c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:32.523014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14ec8d-210e-4b81-9bda-a01fa22e107c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:32.540801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ad14ec8d-210e-4b81-9bda-a01fa22e107c None None] MFA Rules not processed for user `54e3bdd8f01b4ee5be6ac5e6bc44549b`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:32.542005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ad14ec8d-210e-4b81-9bda-a01fa22e107c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:32.576220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:32.598017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1115/2266] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:33.906556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a858c-7060-47f0-bed1-cb07f69567c2 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:33.915982 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a858c-7060-47f0-bed1-cb07f69567c2 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:33.919864 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd0dcb-cbfc-4659-9d35-4aee4041adff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:33.920505 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd0dcb-cbfc-4659-9d35-4aee4041adff service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:33.920789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd0dcb-cbfc-4659-9d35-4aee4041adff service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:33.921067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd0dcb-cbfc-4659-9d35-4aee4041adff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:33.978661 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd0dcb-cbfc-4659-9d35-4aee4041adff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:33.980161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd0dcb-cbfc-4659-9d35-4aee4041adff service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:33.989454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1152/2267] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:10:35.546629 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14ec8d-210e-4b81-9bda-a01fa22e107c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:35.556960 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14ec8d-210e-4b81-9bda-a01fa22e107c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:35.560742 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17169cd4-5f9f-434f-b1ad-ae7d04a7eae1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:35.561394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17169cd4-5f9f-434f-b1ad-ae7d04a7eae1 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:35.561685 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17169cd4-5f9f-434f-b1ad-ae7d04a7eae1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:35.561970 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17169cd4-5f9f-434f-b1ad-ae7d04a7eae1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:35.573455 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17169cd4-5f9f-434f-b1ad-ae7d04a7eae1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:35.575326 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17169cd4-5f9f-434f-b1ad-ae7d04a7eae1 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:35.586835 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1116/2268] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:35 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:10:36.205724 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd0dcb-cbfc-4659-9d35-4aee4041adff service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:36.207230 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b8fd0dcb-cbfc-4659-9d35-4aee4041adff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:36.275811 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:36.279787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd0dcb-cbfc-4659-9d35-4aee4041adff service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:36.286780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166380e9-8b75-4ea7-a9e1-c6ee23a8111c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:36.287125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166380e9-8b75-4ea7-a9e1-c6ee23a8111c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:36.287438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166380e9-8b75-4ea7-a9e1-c6ee23a8111c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:36.287935 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166380e9-8b75-4ea7-a9e1-c6ee23a8111c admin admin] PATH_INFO: `/v3/users/987ef34207774fb183b9d6a1da86893a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:36.289119 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166380e9-8b75-4ea7-a9e1-c6ee23a8111c admin admin] RBAC: Authorizing `identity:delete_user(user_id=987ef34207774fb183b9d6a1da86893a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:36.291268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166380e9-8b75-4ea7-a9e1-c6ee23a8111c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:36.454607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-166380e9-8b75-4ea7-a9e1-c6ee23a8111c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '987ef34207774fb183b9d6a1da86893a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:36.455231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-166380e9-8b75-4ea7-a9e1-c6ee23a8111c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '987ef34207774fb183b9d6a1da86893a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:36.466998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-166380e9-8b75-4ea7-a9e1-c6ee23a8111c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '987ef34207774fb183b9d6a1da86893a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:36.467854 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-166380e9-8b75-4ea7-a9e1-c6ee23a8111c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '987ef34207774fb183b9d6a1da86893a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:36.477027 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-166380e9-8b75-4ea7-a9e1-c6ee23a8111c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '987ef34207774fb183b9d6a1da86893a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:36.567819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1153/2269] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:10:36 2022] DELETE /identity/v3/users/987ef34207774fb183b9d6a1da86893a => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:36.572521 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de9ae8-3f66-4622-aab2-e6191fb28678 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:36.572980 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de9ae8-3f66-4622-aab2-e6191fb28678 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:36.573145 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de9ae8-3f66-4622-aab2-e6191fb28678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:36.592703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-56de9ae8-3f66-4622-aab2-e6191fb28678 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:36.594927 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-56de9ae8-3f66-4622-aab2-e6191fb28678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:36.658957 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1117/2270] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:10:36.668656 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166380e9-8b75-4ea7-a9e1-c6ee23a8111c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:36.670176 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-166380e9-8b75-4ea7-a9e1-c6ee23a8111c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:36.734028 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166380e9-8b75-4ea7-a9e1-c6ee23a8111c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:36.737770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4298c9e-7831-4287-bfb1-994ebc8a2369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:36.738424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4298c9e-7831-4287-bfb1-994ebc8a2369 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:36.738733 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4298c9e-7831-4287-bfb1-994ebc8a2369 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:36.739012 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4298c9e-7831-4287-bfb1-994ebc8a2369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:36.748594 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4298c9e-7831-4287-bfb1-994ebc8a2369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:36.750049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4298c9e-7831-4287-bfb1-994ebc8a2369 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:36.760348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1154/2271] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:10:37.222239 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de9ae8-3f66-4622-aab2-e6191fb28678 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:37.225975 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-56de9ae8-3f66-4622-aab2-e6191fb28678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:37.242987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de9ae8-3f66-4622-aab2-e6191fb28678 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:37.247637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786064c-f9ac-4229-8a8f-5d209603b39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:37.248543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786064c-f9ac-4229-8a8f-5d209603b39b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:37.248790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786064c-f9ac-4229-8a8f-5d209603b39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:37.249135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786064c-f9ac-4229-8a8f-5d209603b39b admin admin] PATH_INFO: `/v3/projects/4cfdae029d3a4d139195ff33254b5bad` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:37.251607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786064c-f9ac-4229-8a8f-5d209603b39b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cfdae029d3a4d139195ff33254b5bad)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:37.253681 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786064c-f9ac-4229-8a8f-5d209603b39b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:37.368560 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0786064c-f9ac-4229-8a8f-5d209603b39b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cfdae029d3a4d139195ff33254b5bad', 'request_id': 'req-0786064c-f9ac-4229-8a8f-5d209603b39b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:37.428826 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0786064c-f9ac-4229-8a8f-5d209603b39b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cfdae029d3a4d139195ff33254b5bad', 'request_id': 'req-0786064c-f9ac-4229-8a8f-5d209603b39b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:37.435040 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0786064c-f9ac-4229-8a8f-5d209603b39b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cfdae029d3a4d139195ff33254b5bad', 'request_id': 'req-0786064c-f9ac-4229-8a8f-5d209603b39b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:37.446890 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0786064c-f9ac-4229-8a8f-5d209603b39b admin admin] The token cache is being invalidate because project 4cfdae029d3a4d139195ff33254b5bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:10:37.452166 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0786064c-f9ac-4229-8a8f-5d209603b39b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:37.452166 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0786064c-f9ac-4229-8a8f-5d209603b39b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:37.452166 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1118/2272] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:10:37 2022] DELETE /identity/v3/projects/4cfdae029d3a4d139195ff33254b5bad => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:37.454934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb2a32-1f9c-4159-9dae-5e2c3bdc82e9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:37.455453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb2a32-1f9c-4159-9dae-5e2c3bdc82e9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:37.455535 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb2a32-1f9c-4159-9dae-5e2c3bdc82e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:37.475466 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-b9bb2a32-1f9c-4159-9dae-5e2c3bdc82e9 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:37.475466 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b9bb2a32-1f9c-4159-9dae-5e2c3bdc82e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:37.483993 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6ddf7-aeee-4fd2-bba2-0c26052cebda None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:37.484297 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6ddf7-aeee-4fd2-bba2-0c26052cebda None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:37.484590 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6ddf7-aeee-4fd2-bba2-0c26052cebda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:37.505819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-5cf6ddf7-aeee-4fd2-bba2-0c26052cebda None None] MFA Rules not processed for user `5aa7ff2fa5454bba9268e89c44b75de6`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:37.507062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5cf6ddf7-aeee-4fd2-bba2-0c26052cebda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:37.563398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1155/2273] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:37.574129 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:37.588813 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1119/2274] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:37.645319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a51f91-9160-441b-a887-a690b34afe22 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:37.645612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a51f91-9160-441b-a887-a690b34afe22 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:37.645892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a51f91-9160-441b-a887-a690b34afe22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:37.677608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e9a51f91-9160-441b-a887-a690b34afe22 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:37.679593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e9a51f91-9160-441b-a887-a690b34afe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:37.692516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1156/2275] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:37.696400 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6ddf7-aeee-4fd2-bba2-0c26052cebda None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:37.708684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6ddf7-aeee-4fd2-bba2-0c26052cebda None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:37.713711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25209c68-29d4-46ec-8681-17d633e06fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:37.714580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25209c68-29d4-46ec-8681-17d633e06fcd admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:37.714872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25209c68-29d4-46ec-8681-17d633e06fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:37.715158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25209c68-29d4-46ec-8681-17d633e06fcd admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:37.716112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25209c68-29d4-46ec-8681-17d633e06fcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:10:37.716427 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25209c68-29d4-46ec-8681-17d633e06fcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:37.718059 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25209c68-29d4-46ec-8681-17d633e06fcd admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:37.738877 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1120/2276] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:10:37.743244 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a51f91-9160-441b-a887-a690b34afe22 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:37.755850 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a51f91-9160-441b-a887-a690b34afe22 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:37.762616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fd123-1965-47ec-96cc-ae797f024267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:37.763413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fd123-1965-47ec-96cc-ae797f024267 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:37.763773 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fd123-1965-47ec-96cc-ae797f024267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:37.764057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fd123-1965-47ec-96cc-ae797f024267 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:37.764946 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9fd123-1965-47ec-96cc-ae797f024267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:37.768163 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9fd123-1965-47ec-96cc-ae797f024267 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:37.779794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:37.901796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1157/2277] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:10:37 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:10:37.905639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25209c68-29d4-46ec-8681-17d633e06fcd admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:37.922065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25209c68-29d4-46ec-8681-17d633e06fcd admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:37.995580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b5e61-1633-4ee4-91c3-8b5fef73c5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:37.996299 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b5e61-1633-4ee4-91c3-8b5fef73c5cb admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:37.996594 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b5e61-1633-4ee4-91c3-8b5fef73c5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:37.996875 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b5e61-1633-4ee4-91c3-8b5fef73c5cb admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:37.997576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b5e61-1633-4ee4-91c3-8b5fef73c5cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:37.999546 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b5e61-1633-4ee4-91c3-8b5fef73c5cb admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:38.099861 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1121/2278] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:10:37 2022] POST /identity/v3/users => generated 291 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:10:38.104612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fd123-1965-47ec-96cc-ae797f024267 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:38.116228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fd123-1965-47ec-96cc-ae797f024267 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:38.126637 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35282860-e956-4c6d-af31-cd09cb761110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:38.127682 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35282860-e956-4c6d-af31-cd09cb761110 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:38.128211 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35282860-e956-4c6d-af31-cd09cb761110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:38.128957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35282860-e956-4c6d-af31-cd09cb761110 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:38.130760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35282860-e956-4c6d-af31-cd09cb761110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:38.133542 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35282860-e956-4c6d-af31-cd09cb761110 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:38.153546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1158/2279] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:10:38.155025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b5e61-1633-4ee4-91c3-8b5fef73c5cb admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:38.168471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b5e61-1633-4ee4-91c3-8b5fef73c5cb admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:38.173700 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1343d-1336-4301-be60-87cf5d085253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:38.174375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1343d-1336-4301-be60-87cf5d085253 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:38.174660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1343d-1336-4301-be60-87cf5d085253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:38.174938 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1343d-1336-4301-be60-87cf5d085253 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:38.175707 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1343d-1336-4301-be60-87cf5d085253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:38.177307 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1343d-1336-4301-be60-87cf5d085253 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:38.192385 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1122/2280] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:10:38.196799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35282860-e956-4c6d-af31-cd09cb761110 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:38.209703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35282860-e956-4c6d-af31-cd09cb761110 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:38.216245 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fd0ef-7d4f-45cc-b9c3-612695ba7415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:38.217158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fd0ef-7d4f-45cc-b9c3-612695ba7415 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:38.217453 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fd0ef-7d4f-45cc-b9c3-612695ba7415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:38.217912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fd0ef-7d4f-45cc-b9c3-612695ba7415 admin admin] PATH_INFO: `/v3/projects/d4d830928ef64831bf3b37562e4bcdf8/users/51c0e902dbb24a928b093f17a2cb1eb0/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:38.236547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32fd0ef-7d4f-45cc-b9c3-612695ba7415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4d830928ef64831bf3b37562e4bcdf8, user_id=51c0e902dbb24a928b093f17a2cb1eb0, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:38.239233 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32fd0ef-7d4f-45cc-b9c3-612695ba7415 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:38.355635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1159/2281] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:10:38 2022] PUT /identity/v3/projects/d4d830928ef64831bf3b37562e4bcdf8/users/51c0e902dbb24a928b093f17a2cb1eb0/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:38.361613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd93c0c-4266-4f8c-b1d6-05f524b9d0a5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:38.362241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd93c0c-4266-4f8c-b1d6-05f524b9d0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:38.362709 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd93c0c-4266-4f8c-b1d6-05f524b9d0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:38.388765 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-9cd93c0c-4266-4f8c-b1d6-05f524b9d0a5 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:38.390416 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9cd93c0c-4266-4f8c-b1d6-05f524b9d0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:38.470444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1123/2282] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:38.476691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fd0ef-7d4f-45cc-b9c3-612695ba7415 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:38.478768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f32fd0ef-7d4f-45cc-b9c3-612695ba7415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:38.553329 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fd0ef-7d4f-45cc-b9c3-612695ba7415 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:38.557185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e406952-7895-4f07-b4ca-53dce40c07a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:38.558043 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e406952-7895-4f07-b4ca-53dce40c07a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:38.558333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e406952-7895-4f07-b4ca-53dce40c07a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:38.558667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e406952-7895-4f07-b4ca-53dce40c07a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:38.571038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e406952-7895-4f07-b4ca-53dce40c07a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:38.572122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e406952-7895-4f07-b4ca-53dce40c07a4 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:38.583373 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1160/2283] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:10:39.955129 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946acc59-6825-4300-ad72-93bc357105b5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:39.955675 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946acc59-6825-4300-ad72-93bc357105b5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:39.955757 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946acc59-6825-4300-ad72-93bc357105b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:39.986514 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-946acc59-6825-4300-ad72-93bc357105b5 None None] MFA Rules not processed for user `51c0e902dbb24a928b093f17a2cb1eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:39.987766 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-946acc59-6825-4300-ad72-93bc357105b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:40.055694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:40.065949 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1124/2284] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:40.076469 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e406952-7895-4f07-b4ca-53dce40c07a4 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:40.078266 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0e406952-7895-4f07-b4ca-53dce40c07a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:40.142191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e406952-7895-4f07-b4ca-53dce40c07a4 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:40.153743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee4702-007c-47a0-95b6-3f6ed78bfd50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:40.153743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee4702-007c-47a0-95b6-3f6ed78bfd50 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:40.153743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee4702-007c-47a0-95b6-3f6ed78bfd50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:40.153743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee4702-007c-47a0-95b6-3f6ed78bfd50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:40.157960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee4702-007c-47a0-95b6-3f6ed78bfd50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:40.159478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee4702-007c-47a0-95b6-3f6ed78bfd50 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:40.170295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1161/2285] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:10:41.486537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946acc59-6825-4300-ad72-93bc357105b5 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:41.487983 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-946acc59-6825-4300-ad72-93bc357105b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:41.501447 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946acc59-6825-4300-ad72-93bc357105b5 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:41.505911 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83160f92-d56a-4264-aef0-53af284c2ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:41.506781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83160f92-d56a-4264-aef0-53af284c2ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:41.507072 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83160f92-d56a-4264-aef0-53af284c2ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:41.507385 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83160f92-d56a-4264-aef0-53af284c2ed9 admin admin] PATH_INFO: `/v3/users/49aca23c1ad24971bd54a411c6c2da18` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:41.508585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83160f92-d56a-4264-aef0-53af284c2ed9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49aca23c1ad24971bd54a411c6c2da18)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:41.510514 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83160f92-d56a-4264-aef0-53af284c2ed9 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:41.677526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-83160f92-d56a-4264-aef0-53af284c2ed9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49aca23c1ad24971bd54a411c6c2da18'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:41.684729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-83160f92-d56a-4264-aef0-53af284c2ed9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49aca23c1ad24971bd54a411c6c2da18'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:41.685512 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-83160f92-d56a-4264-aef0-53af284c2ed9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49aca23c1ad24971bd54a411c6c2da18'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:41.686180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-83160f92-d56a-4264-aef0-53af284c2ed9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49aca23c1ad24971bd54a411c6c2da18'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:41.698401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-83160f92-d56a-4264-aef0-53af284c2ed9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49aca23c1ad24971bd54a411c6c2da18'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:41.761068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1125/2286] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:10:41 2022] DELETE /identity/v3/users/49aca23c1ad24971bd54a411c6c2da18 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:41.765353 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cb471-01d9-475b-9f5c-481a4a9c4659 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:41.765787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cb471-01d9-475b-9f5c-481a4a9c4659 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:41.766506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cb471-01d9-475b-9f5c-481a4a9c4659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:41.789140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-a25cb471-01d9-475b-9f5c-481a4a9c4659 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:41.790348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a25cb471-01d9-475b-9f5c-481a4a9c4659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:41.855610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1162/2287] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:41.866727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83160f92-d56a-4264-aef0-53af284c2ed9 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:41.868415 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-83160f92-d56a-4264-aef0-53af284c2ed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:41.934529 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83160f92-d56a-4264-aef0-53af284c2ed9 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:41.938718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd51fb7-398b-4c04-b7b3-ff195374bd15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:41.939610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd51fb7-398b-4c04-b7b3-ff195374bd15 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:41.940065 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd51fb7-398b-4c04-b7b3-ff195374bd15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:41.940488 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd51fb7-398b-4c04-b7b3-ff195374bd15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:41.952686 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd51fb7-398b-4c04-b7b3-ff195374bd15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:41.954708 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd51fb7-398b-4c04-b7b3-ff195374bd15 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:41.964903 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1126/2288] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:10:42.368620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cb471-01d9-475b-9f5c-481a4a9c4659 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:42.370034 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a25cb471-01d9-475b-9f5c-481a4a9c4659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:42.376991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:42.383376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cb471-01d9-475b-9f5c-481a4a9c4659 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:42.388782 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6151bcad-3708-4dc4-9bbf-b262a3f03c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:42.393014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151bcad-3708-4dc4-9bbf-b262a3f03c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:42.393014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151bcad-3708-4dc4-9bbf-b262a3f03c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:42.393014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151bcad-3708-4dc4-9bbf-b262a3f03c3a admin admin] PATH_INFO: `/v3/projects/45825d43d2ac4b349b7dbee42a0ebc47` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:42.393014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6151bcad-3708-4dc4-9bbf-b262a3f03c3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=45825d43d2ac4b349b7dbee42a0ebc47)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:42.395098 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6151bcad-3708-4dc4-9bbf-b262a3f03c3a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:42.505143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6151bcad-3708-4dc4-9bbf-b262a3f03c3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45825d43d2ac4b349b7dbee42a0ebc47', 'request_id': 'req-6151bcad-3708-4dc4-9bbf-b262a3f03c3a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:42.554477 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6151bcad-3708-4dc4-9bbf-b262a3f03c3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45825d43d2ac4b349b7dbee42a0ebc47', 'request_id': 'req-6151bcad-3708-4dc4-9bbf-b262a3f03c3a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:42.560911 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6151bcad-3708-4dc4-9bbf-b262a3f03c3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45825d43d2ac4b349b7dbee42a0ebc47', 'request_id': 'req-6151bcad-3708-4dc4-9bbf-b262a3f03c3a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:42.574211 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6151bcad-3708-4dc4-9bbf-b262a3f03c3a admin admin] The token cache is being invalidate because project 45825d43d2ac4b349b7dbee42a0ebc47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:10:42.574727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6151bcad-3708-4dc4-9bbf-b262a3f03c3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:42.575823 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6151bcad-3708-4dc4-9bbf-b262a3f03c3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:42.578969 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1163/2289] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:10:42 2022] DELETE /identity/v3/projects/45825d43d2ac4b349b7dbee42a0ebc47 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:42.582398 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd51fb7-398b-4c04-b7b3-ff195374bd15 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:42.592130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd51fb7-398b-4c04-b7b3-ff195374bd15 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:42.653295 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab185a-b220-4ebe-b304-55af37e66869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:42.654360 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab185a-b220-4ebe-b304-55af37e66869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:42.654645 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab185a-b220-4ebe-b304-55af37e66869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:42.654923 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab185a-b220-4ebe-b304-55af37e66869 admin admin] PATH_INFO: `/v3/users/4a46abfb1edf4d0c9c097923f933e09f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:42.656215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab185a-b220-4ebe-b304-55af37e66869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a46abfb1edf4d0c9c097923f933e09f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:42.658172 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab185a-b220-4ebe-b304-55af37e66869 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:42.671008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:42.825890 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-46ab185a-b220-4ebe-b304-55af37e66869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a46abfb1edf4d0c9c097923f933e09f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:42.832822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-46ab185a-b220-4ebe-b304-55af37e66869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a46abfb1edf4d0c9c097923f933e09f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:42.833648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-46ab185a-b220-4ebe-b304-55af37e66869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a46abfb1edf4d0c9c097923f933e09f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:42.834717 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-46ab185a-b220-4ebe-b304-55af37e66869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a46abfb1edf4d0c9c097923f933e09f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:42.847398 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-46ab185a-b220-4ebe-b304-55af37e66869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a46abfb1edf4d0c9c097923f933e09f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:42.930253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1127/2290] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:10:42 2022] DELETE /identity/v3/users/4a46abfb1edf4d0c9c097923f933e09f => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:42.986581 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cca1a-0cb3-48ad-a820-66386a4c5c42 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:42.987724 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cca1a-0cb3-48ad-a820-66386a4c5c42 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:42.988210 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cca1a-0cb3-48ad-a820-66386a4c5c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:43.029988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-1b3cca1a-0cb3-48ad-a820-66386a4c5c42 None None] MFA Rules not processed for user `51c0e902dbb24a928b093f17a2cb1eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:43.032780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1b3cca1a-0cb3-48ad-a820-66386a4c5c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:43.103211 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1164/2291] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:43.653843 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab185a-b220-4ebe-b304-55af37e66869 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:43.658324 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-46ab185a-b220-4ebe-b304-55af37e66869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:43.759085 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab185a-b220-4ebe-b304-55af37e66869 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:43.763787 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b653db4-61ae-48bc-a213-62a5bcd8affb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:43.764819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b653db4-61ae-48bc-a213-62a5bcd8affb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:43.765841 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b653db4-61ae-48bc-a213-62a5bcd8affb admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:43.765841 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b653db4-61ae-48bc-a213-62a5bcd8affb admin admin] PATH_INFO: `/v3/projects/883e09cb29de4f85a78ae6304a495d12` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:43.767818 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b653db4-61ae-48bc-a213-62a5bcd8affb admin admin] RBAC: Authorizing `identity:delete_project(project_id=883e09cb29de4f85a78ae6304a495d12)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:43.770438 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b653db4-61ae-48bc-a213-62a5bcd8affb admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:43.947072 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3b653db4-61ae-48bc-a213-62a5bcd8affb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '883e09cb29de4f85a78ae6304a495d12', 'request_id': 'req-3b653db4-61ae-48bc-a213-62a5bcd8affb'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:44.058160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3b653db4-61ae-48bc-a213-62a5bcd8affb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '883e09cb29de4f85a78ae6304a495d12', 'request_id': 'req-3b653db4-61ae-48bc-a213-62a5bcd8affb'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:44.069434 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3b653db4-61ae-48bc-a213-62a5bcd8affb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '883e09cb29de4f85a78ae6304a495d12', 'request_id': 'req-3b653db4-61ae-48bc-a213-62a5bcd8affb'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:44.117296 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3b653db4-61ae-48bc-a213-62a5bcd8affb admin admin] The token cache is being invalidate because project 883e09cb29de4f85a78ae6304a495d12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:10:44.117777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3b653db4-61ae-48bc-a213-62a5bcd8affb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:44.119069 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3b653db4-61ae-48bc-a213-62a5bcd8affb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:44.132563 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cca1a-0cb3-48ad-a820-66386a4c5c42 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:44.133096 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1128/2292] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:10:43 2022] DELETE /identity/v3/projects/883e09cb29de4f85a78ae6304a495d12 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:44.147112 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cca1a-0cb3-48ad-a820-66386a4c5c42 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:44.224143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79abd5-7006-4767-8274-b7121378c8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:44.226306 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79abd5-7006-4767-8274-b7121378c8f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:44.226773 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79abd5-7006-4767-8274-b7121378c8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:44.227328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79abd5-7006-4767-8274-b7121378c8f8 admin admin] PATH_INFO: `/v3/users/54e280b2bb1545c6806ae9a59b608d30` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:44.229983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a79abd5-7006-4767-8274-b7121378c8f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54e280b2bb1545c6806ae9a59b608d30)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:44.232577 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a79abd5-7006-4767-8274-b7121378c8f8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:44.425381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:44.439369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a79abd5-7006-4767-8274-b7121378c8f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54e280b2bb1545c6806ae9a59b608d30'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:44.440477 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a79abd5-7006-4767-8274-b7121378c8f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54e280b2bb1545c6806ae9a59b608d30'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:44.465971 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a79abd5-7006-4767-8274-b7121378c8f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54e280b2bb1545c6806ae9a59b608d30'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:44.466721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a79abd5-7006-4767-8274-b7121378c8f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54e280b2bb1545c6806ae9a59b608d30'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:44.476783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a79abd5-7006-4767-8274-b7121378c8f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54e280b2bb1545c6806ae9a59b608d30'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:44.602434 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1165/2293] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:10:44 2022] DELETE /identity/v3/users/54e280b2bb1545c6806ae9a59b608d30 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:45.163926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b653db4-61ae-48bc-a213-62a5bcd8affb admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:45.165645 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3b653db4-61ae-48bc-a213-62a5bcd8affb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:45.200346 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b653db4-61ae-48bc-a213-62a5bcd8affb admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:45.219677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:45.265271 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8406052-0055-4403-84f6-f8180c018ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:45.266321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8406052-0055-4403-84f6-f8180c018ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:45.266706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8406052-0055-4403-84f6-f8180c018ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:45.267098 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8406052-0055-4403-84f6-f8180c018ee5 admin admin] PATH_INFO: `/v3/projects/b8dfb078c0644c659cf2f2ef4d7a0c46` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:45.268905 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8406052-0055-4403-84f6-f8180c018ee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8dfb078c0644c659cf2f2ef4d7a0c46)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:45.271691 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8406052-0055-4403-84f6-f8180c018ee5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:45.424857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e8406052-0055-4403-84f6-f8180c018ee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8dfb078c0644c659cf2f2ef4d7a0c46', 'request_id': 'req-e8406052-0055-4403-84f6-f8180c018ee5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:45.526309 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e8406052-0055-4403-84f6-f8180c018ee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8dfb078c0644c659cf2f2ef4d7a0c46', 'request_id': 'req-e8406052-0055-4403-84f6-f8180c018ee5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:45.533829 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e8406052-0055-4403-84f6-f8180c018ee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8dfb078c0644c659cf2f2ef4d7a0c46', 'request_id': 'req-e8406052-0055-4403-84f6-f8180c018ee5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:45.545448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e8406052-0055-4403-84f6-f8180c018ee5 admin admin] The token cache is being invalidate because project b8dfb078c0644c659cf2f2ef4d7a0c46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:10:45.545784 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e8406052-0055-4403-84f6-f8180c018ee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:45.546476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e8406052-0055-4403-84f6-f8180c018ee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:10:45.548706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1129/2294] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:10:45 2022] DELETE /identity/v3/projects/b8dfb078c0644c659cf2f2ef4d7a0c46 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:45.558144 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115f863-9d3d-403f-9fe0-150a95553774 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:45.558630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115f863-9d3d-403f-9fe0-150a95553774 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:45.558805 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115f863-9d3d-403f-9fe0-150a95553774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:45.579046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-8115f863-9d3d-403f-9fe0-150a95553774 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:45.580401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8115f863-9d3d-403f-9fe0-150a95553774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:45.651428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1166/2295] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:45.660135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df88836-19e3-40e3-8ab4-3627bb2c1a1f None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:45.660562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df88836-19e3-40e3-8ab4-3627bb2c1a1f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:45.662767 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df88836-19e3-40e3-8ab4-3627bb2c1a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:45.689409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-6df88836-19e3-40e3-8ab4-3627bb2c1a1f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:45.690582 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6df88836-19e3-40e3-8ab4-3627bb2c1a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:45.698893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1130/2296] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:45.703005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115f863-9d3d-403f-9fe0-150a95553774 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:45.712977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115f863-9d3d-403f-9fe0-150a95553774 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:45.717957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0aa2b-e31f-4fff-abbd-fc95b43d86d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:45.718629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0aa2b-e31f-4fff-abbd-fc95b43d86d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:45.718918 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0aa2b-e31f-4fff-abbd-fc95b43d86d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:45.719382 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0aa2b-e31f-4fff-abbd-fc95b43d86d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:45.721062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea0aa2b-e31f-4fff-abbd-fc95b43d86d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:10:45.721062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea0aa2b-e31f-4fff-abbd-fc95b43d86d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:45.722038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea0aa2b-e31f-4fff-abbd-fc95b43d86d6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:45.740736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1167/2297] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:10:45.754410 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df88836-19e3-40e3-8ab4-3627bb2c1a1f None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:45.764806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df88836-19e3-40e3-8ab4-3627bb2c1a1f None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:45.769652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b00cbd-a197-419e-a3aa-195a562140d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:45.770411 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b00cbd-a197-419e-a3aa-195a562140d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:45.770769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b00cbd-a197-419e-a3aa-195a562140d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:45.770993 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b00cbd-a197-419e-a3aa-195a562140d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:45.771739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b00cbd-a197-419e-a3aa-195a562140d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:45.774000 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b00cbd-a197-419e-a3aa-195a562140d7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:45.895411 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1131/2298] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:10:45 2022] POST /identity/v3/projects => generated 369 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:10:45.899990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0aa2b-e31f-4fff-abbd-fc95b43d86d6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:45.913683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0aa2b-e31f-4fff-abbd-fc95b43d86d6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:45.972694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101085d9-eb30-46f8-bb3e-3e6e0ddbcc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:45.973332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101085d9-eb30-46f8-bb3e-3e6e0ddbcc71 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:45.973622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101085d9-eb30-46f8-bb3e-3e6e0ddbcc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:45.973907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101085d9-eb30-46f8-bb3e-3e6e0ddbcc71 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:45.974617 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101085d9-eb30-46f8-bb3e-3e6e0ddbcc71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:45.976606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101085d9-eb30-46f8-bb3e-3e6e0ddbcc71 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:46.098475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1168/2299] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:10:45 2022] POST /identity/v3/users => generated 285 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:10:46.102513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b00cbd-a197-419e-a3aa-195a562140d7 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:46.113258 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b00cbd-a197-419e-a3aa-195a562140d7 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:46.117845 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fec0d7-505b-4c75-b938-84cbc4c95560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:46.118499 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fec0d7-505b-4c75-b938-84cbc4c95560 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:46.118782 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fec0d7-505b-4c75-b938-84cbc4c95560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:46.119097 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fec0d7-505b-4c75-b938-84cbc4c95560 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:46.119878 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fec0d7-505b-4c75-b938-84cbc4c95560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:46.121652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fec0d7-505b-4c75-b938-84cbc4c95560 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:46.133916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1132/2300] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:10:46.138395 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101085d9-eb30-46f8-bb3e-3e6e0ddbcc71 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:46.149277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101085d9-eb30-46f8-bb3e-3e6e0ddbcc71 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:46.174924 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6d231-9dbd-4e5c-a615-f98f15a9542d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:46.175946 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6d231-9dbd-4e5c-a615-f98f15a9542d admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:46.176431 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6d231-9dbd-4e5c-a615-f98f15a9542d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:46.176720 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6d231-9dbd-4e5c-a615-f98f15a9542d admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:46.177493 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6d231-9dbd-4e5c-a615-f98f15a9542d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:46.179210 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6d231-9dbd-4e5c-a615-f98f15a9542d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:46.191964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1169/2301] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:10:46.196172 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fec0d7-505b-4c75-b938-84cbc4c95560 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:46.206950 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fec0d7-505b-4c75-b938-84cbc4c95560 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:46.218848 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936bae6-b58a-4895-8ae2-a8fc3d58c67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:46.221157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936bae6-b58a-4895-8ae2-a8fc3d58c67a admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:46.221689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936bae6-b58a-4895-8ae2-a8fc3d58c67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:46.222234 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936bae6-b58a-4895-8ae2-a8fc3d58c67a admin admin] PATH_INFO: `/v3/projects/d8788a3a445c4d1799024476afd11d2a/users/93d160e94f244835ace0e224e57804bb/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:46.400049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5936bae6-b58a-4895-8ae2-a8fc3d58c67a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8788a3a445c4d1799024476afd11d2a, user_id=93d160e94f244835ace0e224e57804bb, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:46.400049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5936bae6-b58a-4895-8ae2-a8fc3d58c67a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:46.744612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1133/2302] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:10:46 2022] PUT /identity/v3/projects/d8788a3a445c4d1799024476afd11d2a/users/93d160e94f244835ace0e224e57804bb/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:10:46.758029 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3e13b-34c6-463e-86aa-23a14ae6ba35 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:46.758634 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3e13b-34c6-463e-86aa-23a14ae6ba35 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:46.758634 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3e13b-34c6-463e-86aa-23a14ae6ba35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:46.794581 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3ea3e13b-34c6-463e-86aa-23a14ae6ba35 None None] MFA Rules not processed for user `93d160e94f244835ace0e224e57804bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:46.794581 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3ea3e13b-34c6-463e-86aa-23a14ae6ba35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:46.800099 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:46.879304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1170/2303] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:46 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:46.890756 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936bae6-b58a-4895-8ae2-a8fc3d58c67a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:10:46.893145 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5936bae6-b58a-4895-8ae2-a8fc3d58c67a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:46.955369 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936bae6-b58a-4895-8ae2-a8fc3d58c67a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:10:46.959249 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5f6a2-c863-4215-bb1e-b2e59e86931f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5d76eae62b54521a95f36284118fada', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:10:46.962894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5f6a2-c863-4215-bb1e-b2e59e86931f service glance] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:46.962894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5f6a2-c863-4215-bb1e-b2e59e86931f service glance] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:46.962894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5f6a2-c863-4215-bb1e-b2e59e86931f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:46.969648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5f6a2-c863-4215-bb1e-b2e59e86931f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:10:46.971304 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5f6a2-c863-4215-bb1e-b2e59e86931f service glance] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:10:46.983733 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1134/2304] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:10:46 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:10:47.812739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:10:57.449617 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f9407-4940-42a8-a3c8-3a41319df130 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:10:57.450055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f9407-4940-42a8-a3c8-3a41319df130 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:10:57.450316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f9407-4940-42a8-a3c8-3a41319df130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:10:57.469372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-724f9407-4940-42a8-a3c8-3a41319df130 None None] MFA Rules not processed for user `54e3bdd8f01b4ee5be6ac5e6bc44549b`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:10:57.471515 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-724f9407-4940-42a8-a3c8-3a41319df130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:10:57.542896 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1171/2305] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:10:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:10:58.428762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:11:01.169351 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5f6a2-c863-4215-bb1e-b2e59e86931f service glance] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:11:01.170811 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7eb5f6a2-c863-4215-bb1e-b2e59e86931f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:11:01.243879 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5f6a2-c863-4215-bb1e-b2e59e86931f service glance] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:11:01.249037 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bb1dc-17e4-4a94-884b-ace722ef5166 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c577fa8670e34449a80de0c8e0a15a8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:11:01.249698 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bb1dc-17e4-4a94-884b-ace722ef5166 service placement] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:01.249991 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bb1dc-17e4-4a94-884b-ace722ef5166 service placement] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:01.250275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bb1dc-17e4-4a94-884b-ace722ef5166 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:01.252732 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4a0bb1dc-17e4-4a94-884b-ace722ef5166 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:11:01.265471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0bb1dc-17e4-4a94-884b-ace722ef5166 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:11:01.267706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0bb1dc-17e4-4a94-884b-ace722ef5166 service placement] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:11:01.279045 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1135/2306] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:11:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:11:07.535644 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=50 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:11:11.274705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=32 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:11:30.272910 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f9407-4940-42a8-a3c8-3a41319df130 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:11:30.335276 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f9407-4940-42a8-a3c8-3a41319df130 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:11:30.339924 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d9792-59cd-4f67-8aad-e245aba2fdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:11:30.340795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d9792-59cd-4f67-8aad-e245aba2fdcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:30.341081 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d9792-59cd-4f67-8aad-e245aba2fdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:30.341362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d9792-59cd-4f67-8aad-e245aba2fdcd admin admin] PATH_INFO: `/v3/users/93d160e94f244835ace0e224e57804bb` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:30.342599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d9792-59cd-4f67-8aad-e245aba2fdcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=93d160e94f244835ace0e224e57804bb)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:11:30.344555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d9792-59cd-4f67-8aad-e245aba2fdcd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:11:30.513198 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-876d9792-59cd-4f67-8aad-e245aba2fdcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93d160e94f244835ace0e224e57804bb'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:30.513198 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-876d9792-59cd-4f67-8aad-e245aba2fdcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93d160e94f244835ace0e224e57804bb'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:30.527107 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-876d9792-59cd-4f67-8aad-e245aba2fdcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93d160e94f244835ace0e224e57804bb'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:30.527966 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-876d9792-59cd-4f67-8aad-e245aba2fdcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93d160e94f244835ace0e224e57804bb'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:30.537783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-876d9792-59cd-4f67-8aad-e245aba2fdcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93d160e94f244835ace0e224e57804bb'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:30.611668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1172/2307] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:11:30 2022] DELETE /identity/v3/users/93d160e94f244835ace0e224e57804bb => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:11:30.616854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66054b1a-4614-46cc-90fb-51f120421fa3 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:30.617368 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66054b1a-4614-46cc-90fb-51f120421fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:30.617476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66054b1a-4614-46cc-90fb-51f120421fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:30.635280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-66054b1a-4614-46cc-90fb-51f120421fa3 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:11:30.639681 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-66054b1a-4614-46cc-90fb-51f120421fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:11:30.697666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1136/2308] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:11:30.708534 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d9792-59cd-4f67-8aad-e245aba2fdcd admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:11:30.710294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-876d9792-59cd-4f67-8aad-e245aba2fdcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:11:30.776040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d9792-59cd-4f67-8aad-e245aba2fdcd admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:11:30.779769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b832d70-c09c-4e1f-8430-86cb09967b5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:11:30.780375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b832d70-c09c-4e1f-8430-86cb09967b5e service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:30.780657 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b832d70-c09c-4e1f-8430-86cb09967b5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:30.780930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b832d70-c09c-4e1f-8430-86cb09967b5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:30.791789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b832d70-c09c-4e1f-8430-86cb09967b5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:11:30.794955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b832d70-c09c-4e1f-8430-86cb09967b5e service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:11:30.808166 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1173/2309] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:11:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:11:31.207843 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66054b1a-4614-46cc-90fb-51f120421fa3 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:11:31.209546 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-66054b1a-4614-46cc-90fb-51f120421fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:11:31.225980 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66054b1a-4614-46cc-90fb-51f120421fa3 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:11:31.231093 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ce955-6f4d-423d-be40-3f4613af07c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:11:31.232776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ce955-6f4d-423d-be40-3f4613af07c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:31.233075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ce955-6f4d-423d-be40-3f4613af07c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:31.233348 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ce955-6f4d-423d-be40-3f4613af07c8 admin admin] PATH_INFO: `/v3/projects/d8788a3a445c4d1799024476afd11d2a` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:31.234937 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ce955-6f4d-423d-be40-3f4613af07c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8788a3a445c4d1799024476afd11d2a)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:11:31.237024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ce955-6f4d-423d-be40-3f4613af07c8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:11:31.336050 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-736ce955-6f4d-423d-be40-3f4613af07c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8788a3a445c4d1799024476afd11d2a', 'request_id': 'req-736ce955-6f4d-423d-be40-3f4613af07c8'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:31.388782 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-736ce955-6f4d-423d-be40-3f4613af07c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8788a3a445c4d1799024476afd11d2a', 'request_id': 'req-736ce955-6f4d-423d-be40-3f4613af07c8'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:31.395639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-736ce955-6f4d-423d-be40-3f4613af07c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8788a3a445c4d1799024476afd11d2a', 'request_id': 'req-736ce955-6f4d-423d-be40-3f4613af07c8'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:31.408335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-736ce955-6f4d-423d-be40-3f4613af07c8 admin admin] The token cache is being invalidate because project d8788a3a445c4d1799024476afd11d2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:11:31.408645 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-736ce955-6f4d-423d-be40-3f4613af07c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:31.409292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-736ce955-6f4d-423d-be40-3f4613af07c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:31.411253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1137/2310] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:11:31 2022] DELETE /identity/v3/projects/d8788a3a445c4d1799024476afd11d2a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:11:31.415657 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62441a-d273-4b5c-a098-58fbffc39195 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:31.416159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62441a-d273-4b5c-a098-58fbffc39195 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:31.416264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62441a-d273-4b5c-a098-58fbffc39195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:31.434279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6c62441a-d273-4b5c-a098-58fbffc39195 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:11:31.435506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6c62441a-d273-4b5c-a098-58fbffc39195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:11:31.453375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:11:31.498311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1174/2311] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:11:31.506599 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79bcf1-5a37-4c16-87e3-4ca8f641d950 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:31.506893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79bcf1-5a37-4c16-87e3-4ca8f641d950 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:31.507192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79bcf1-5a37-4c16-87e3-4ca8f641d950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:31.525509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7b79bcf1-5a37-4c16-87e3-4ca8f641d950 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:11:31.526703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7b79bcf1-5a37-4c16-87e3-4ca8f641d950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:11:31.534869 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1138/2312] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:11:31.539104 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62441a-d273-4b5c-a098-58fbffc39195 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:11:31.548695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62441a-d273-4b5c-a098-58fbffc39195 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:11:31.553677 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7671fa5-87b4-44ad-8403-2fcc77c59ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:11:31.554321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7671fa5-87b4-44ad-8403-2fcc77c59ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:31.554610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7671fa5-87b4-44ad-8403-2fcc77c59ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:31.554887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7671fa5-87b4-44ad-8403-2fcc77c59ec6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:31.555645 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7671fa5-87b4-44ad-8403-2fcc77c59ec6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:11:31.555951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7671fa5-87b4-44ad-8403-2fcc77c59ec6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:11:31.557548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7671fa5-87b4-44ad-8403-2fcc77c59ec6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:11:31.569598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1175/2313] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:11:31.573602 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79bcf1-5a37-4c16-87e3-4ca8f641d950 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:11:31.582900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79bcf1-5a37-4c16-87e3-4ca8f641d950 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:11:31.587519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd67cac-469c-41cf-9ca6-3a52e07e32a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:11:31.588162 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd67cac-469c-41cf-9ca6-3a52e07e32a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:31.588447 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd67cac-469c-41cf-9ca6-3a52e07e32a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:31.588722 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd67cac-469c-41cf-9ca6-3a52e07e32a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:31.589420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd67cac-469c-41cf-9ca6-3a52e07e32a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:11:31.591588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd67cac-469c-41cf-9ca6-3a52e07e32a4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:11:31.676568 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1139/2314] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:11:31 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:11:31.680713 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7671fa5-87b4-44ad-8403-2fcc77c59ec6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:11:31.690551 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7671fa5-87b4-44ad-8403-2fcc77c59ec6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:11:31.747155 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3768b-c84a-42cf-a003-c81f25e62815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:11:31.747825 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3768b-c84a-42cf-a003-c81f25e62815 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:31.748114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3768b-c84a-42cf-a003-c81f25e62815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:31.748395 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3768b-c84a-42cf-a003-c81f25e62815 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:31.749091 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3768b-c84a-42cf-a003-c81f25e62815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:11:31.751018 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3768b-c84a-42cf-a003-c81f25e62815 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:11:31.846630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1176/2315] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:11:31 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:11:31.850602 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd67cac-469c-41cf-9ca6-3a52e07e32a4 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:11:31.860836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd67cac-469c-41cf-9ca6-3a52e07e32a4 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:11:31.865954 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736191b-3ed8-44e5-9ec1-8b011a5e80d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:11:31.866609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736191b-3ed8-44e5-9ec1-8b011a5e80d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:31.866913 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736191b-3ed8-44e5-9ec1-8b011a5e80d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:31.867218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736191b-3ed8-44e5-9ec1-8b011a5e80d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:31.867942 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736191b-3ed8-44e5-9ec1-8b011a5e80d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:11:31.869531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736191b-3ed8-44e5-9ec1-8b011a5e80d9 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:11:31.880791 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1140/2316] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:11:31.884784 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3768b-c84a-42cf-a003-c81f25e62815 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:11:31.894813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3768b-c84a-42cf-a003-c81f25e62815 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:11:31.899765 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38503a37-a747-4ef7-b358-8da6daf64d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:11:31.900425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38503a37-a747-4ef7-b358-8da6daf64d92 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:31.900708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38503a37-a747-4ef7-b358-8da6daf64d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:31.900982 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38503a37-a747-4ef7-b358-8da6daf64d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:31.901700 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38503a37-a747-4ef7-b358-8da6daf64d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:11:31.903326 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38503a37-a747-4ef7-b358-8da6daf64d92 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:11:31.913848 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1177/2317] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:11:31.918001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736191b-3ed8-44e5-9ec1-8b011a5e80d9 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:11:31.928221 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736191b-3ed8-44e5-9ec1-8b011a5e80d9 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:11:31.932957 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5c573-c5ba-4375-bf95-d544b8f6c29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:11:31.933830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5c573-c5ba-4375-bf95-d544b8f6c29b admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:31.934112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5c573-c5ba-4375-bf95-d544b8f6c29b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:31.934387 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5c573-c5ba-4375-bf95-d544b8f6c29b admin admin] PATH_INFO: `/v3/projects/73b674367a2c4c68bb00155e9f9d5f24/users/e076108ae0374ee282bb02c7b5eac315/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:31.949115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5c573-c5ba-4375-bf95-d544b8f6c29b admin admin] RBAC: Authorizing `identity:create_grant(project_id=73b674367a2c4c68bb00155e9f9d5f24, user_id=e076108ae0374ee282bb02c7b5eac315, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:11:31.952017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5c573-c5ba-4375-bf95-d544b8f6c29b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:11:32.017368 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1141/2318] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:11:31 2022] PUT /identity/v3/projects/73b674367a2c4c68bb00155e9f9d5f24/users/e076108ae0374ee282bb02c7b5eac315/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:11:32.084191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda5d6f-0a2d-4608-9632-376071858fce None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:32.084489 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda5d6f-0a2d-4608-9632-376071858fce None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:32.084778 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda5d6f-0a2d-4608-9632-376071858fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:32.112935 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5cda5d6f-0a2d-4608-9632-376071858fce None None] MFA Rules not processed for user `e076108ae0374ee282bb02c7b5eac315`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:11:32.114186 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5cda5d6f-0a2d-4608-9632-376071858fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:11:32.178830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1178/2319] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:11:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:11:32.190247 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5c573-c5ba-4375-bf95-d544b8f6c29b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:11:32.191948 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-99b5c573-c5ba-4375-bf95-d544b8f6c29b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:11:32.213542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:11:32.247894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5c573-c5ba-4375-bf95-d544b8f6c29b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:11:32.252196 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ca8d0-6085-40d5-b624-75fe034de3bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:11:32.252894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ca8d0-6085-40d5-b624-75fe034de3bf service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:32.253179 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ca8d0-6085-40d5-b624-75fe034de3bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:32.253455 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ca8d0-6085-40d5-b624-75fe034de3bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:32.261603 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ca8d0-6085-40d5-b624-75fe034de3bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:11:32.263014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ca8d0-6085-40d5-b624-75fe034de3bf service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:11:32.272449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1142/2320] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:11:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:11:32.298128 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda5d6f-0a2d-4608-9632-376071858fce None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:11:32.317431 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:11:32.360059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda5d6f-0a2d-4608-9632-376071858fce None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:11:32.364953 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e71c2-5954-4837-b057-d704df48fc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:11:32.365806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e71c2-5954-4837-b057-d704df48fc01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:32.366114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e71c2-5954-4837-b057-d704df48fc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:32.366390 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e71c2-5954-4837-b057-d704df48fc01 admin admin] PATH_INFO: `/v3/users/e076108ae0374ee282bb02c7b5eac315` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:32.367646 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e71c2-5954-4837-b057-d704df48fc01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e076108ae0374ee282bb02c7b5eac315)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:11:32.369748 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e71c2-5954-4837-b057-d704df48fc01 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:11:32.564131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-674e71c2-5954-4837-b057-d704df48fc01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e076108ae0374ee282bb02c7b5eac315'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:32.565738 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-674e71c2-5954-4837-b057-d704df48fc01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e076108ae0374ee282bb02c7b5eac315'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:32.578826 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-674e71c2-5954-4837-b057-d704df48fc01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e076108ae0374ee282bb02c7b5eac315'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:32.579660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-674e71c2-5954-4837-b057-d704df48fc01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e076108ae0374ee282bb02c7b5eac315'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:32.588119 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-674e71c2-5954-4837-b057-d704df48fc01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e076108ae0374ee282bb02c7b5eac315'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:32.652562 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1179/2321] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:11:32 2022] DELETE /identity/v3/users/e076108ae0374ee282bb02c7b5eac315 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:11:32.656823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c7bd2-7276-441e-a31c-92200c378d5e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:32.657197 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c7bd2-7276-441e-a31c-92200c378d5e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:32.657554 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c7bd2-7276-441e-a31c-92200c378d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:32.676043 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d46c7bd2-7276-441e-a31c-92200c378d5e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:11:32.682483 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d46c7bd2-7276-441e-a31c-92200c378d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:11:32.736064 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1143/2322] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:11:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:11:32.745716 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e71c2-5954-4837-b057-d704df48fc01 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:11:32.747151 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-674e71c2-5954-4837-b057-d704df48fc01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:11:32.806552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e71c2-5954-4837-b057-d704df48fc01 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:11:32.810536 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db8c0f-7ab5-42b1-8e5c-d7ebf539f718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:11:32.811205 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db8c0f-7ab5-42b1-8e5c-d7ebf539f718 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:32.811502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db8c0f-7ab5-42b1-8e5c-d7ebf539f718 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:32.811788 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db8c0f-7ab5-42b1-8e5c-d7ebf539f718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:32.820960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db8c0f-7ab5-42b1-8e5c-d7ebf539f718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:11:32.827049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db8c0f-7ab5-42b1-8e5c-d7ebf539f718 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:11:32.833610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1180/2323] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:11:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:11:33.230525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c7bd2-7276-441e-a31c-92200c378d5e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:11:33.232200 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d46c7bd2-7276-441e-a31c-92200c378d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:11:33.250885 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c7bd2-7276-441e-a31c-92200c378d5e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:11:33.254918 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0fa99-ab76-401d-8e40-5d2704a26a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:11:33.256752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0fa99-ab76-401d-8e40-5d2704a26a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:33.257786 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0fa99-ab76-401d-8e40-5d2704a26a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:33.258410 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0fa99-ab76-401d-8e40-5d2704a26a57 admin admin] PATH_INFO: `/v3/projects/73b674367a2c4c68bb00155e9f9d5f24` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:33.259831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0fa99-ab76-401d-8e40-5d2704a26a57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73b674367a2c4c68bb00155e9f9d5f24)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:11:33.261936 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0fa99-ab76-401d-8e40-5d2704a26a57 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:11:33.365596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9be0fa99-ab76-401d-8e40-5d2704a26a57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73b674367a2c4c68bb00155e9f9d5f24', 'request_id': 'req-9be0fa99-ab76-401d-8e40-5d2704a26a57'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:33.422471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9be0fa99-ab76-401d-8e40-5d2704a26a57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73b674367a2c4c68bb00155e9f9d5f24', 'request_id': 'req-9be0fa99-ab76-401d-8e40-5d2704a26a57'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:33.429054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9be0fa99-ab76-401d-8e40-5d2704a26a57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73b674367a2c4c68bb00155e9f9d5f24', 'request_id': 'req-9be0fa99-ab76-401d-8e40-5d2704a26a57'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:33.439995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9be0fa99-ab76-401d-8e40-5d2704a26a57 admin admin] The token cache is being invalidate because project 73b674367a2c4c68bb00155e9f9d5f24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:11:33.440284 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9be0fa99-ab76-401d-8e40-5d2704a26a57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:33.440919 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9be0fa99-ab76-401d-8e40-5d2704a26a57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:33.442466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1144/2324] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:11:33 2022] DELETE /identity/v3/projects/73b674367a2c4c68bb00155e9f9d5f24 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:11:37.755050 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9d9fa-0af9-4115-8bf1-613249b3f983 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:37.755580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9d9fa-0af9-4115-8bf1-613249b3f983 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:37.755675 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9d9fa-0af9-4115-8bf1-613249b3f983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:37.773095 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-08d9d9fa-0af9-4115-8bf1-613249b3f983 None None] MFA Rules not processed for user `4ef0fad9a71d43bfbaba9c87dfed41c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:11:37.774530 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-08d9d9fa-0af9-4115-8bf1-613249b3f983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:11:37.807264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:11:37.828851 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1181/2325] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 14 19:11:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:11:37.873124 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0fa99-ab76-401d-8e40-5d2704a26a57 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:11:37.874341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9be0fa99-ab76-401d-8e40-5d2704a26a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:11:37.923822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:11:37.952775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0fa99-ab76-401d-8e40-5d2704a26a57 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:11:37.958584 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12dcf7-6522-4f16-b6eb-b3a25a5b70cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4368f68724f946eca53084a9ec7ff590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:11:37.959642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12dcf7-6522-4f16-b6eb-b3a25a5b70cd service swift] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:37.960140 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12dcf7-6522-4f16-b6eb-b3a25a5b70cd service swift] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:37.960462 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12dcf7-6522-4f16-b6eb-b3a25a5b70cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:37.969698 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12dcf7-6522-4f16-b6eb-b3a25a5b70cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:11:37.971360 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12dcf7-6522-4f16-b6eb-b3a25a5b70cd service swift] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:11:37.979297 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1145/2326] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 14 19:11:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 19:11:47.890814 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=7 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:11:48.097228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=11 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:11:55.859080 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732f5ff-56d4-4ef8-8261-2ffe5f1cbd6e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:55.859742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732f5ff-56d4-4ef8-8261-2ffe5f1cbd6e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:55.859742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732f5ff-56d4-4ef8-8261-2ffe5f1cbd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:55.881498 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-b732f5ff-56d4-4ef8-8261-2ffe5f1cbd6e None None] MFA Rules not processed for user `4ef0fad9a71d43bfbaba9c87dfed41c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:11:55.882780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b732f5ff-56d4-4ef8-8261-2ffe5f1cbd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:11:55.892103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1182/2327] 10.222.0.69 () {58 vars in 967 bytes} [Thu Jul 14 19:11:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:11:55.907309 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12dcf7-6522-4f16-b6eb-b3a25a5b70cd service swift] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:11:55.918471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12dcf7-6522-4f16-b6eb-b3a25a5b70cd service swift] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:11:55.925136 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdc3c5-87b9-4f60-9130-1251c39e78f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4368f68724f946eca53084a9ec7ff590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:11:55.925985 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdc3c5-87b9-4f60-9130-1251c39e78f3 service swift] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:55.926407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdc3c5-87b9-4f60-9130-1251c39e78f3 service swift] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:55.926812 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdc3c5-87b9-4f60-9130-1251c39e78f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:55.936420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cdc3c5-87b9-4f60-9130-1251c39e78f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:11:55.938155 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cdc3c5-87b9-4f60-9130-1251c39e78f3 service swift] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:11:55.948405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1146/2328] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 14 19:11:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 19:11:59.040138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732f5ff-56d4-4ef8-8261-2ffe5f1cbd6e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:11:59.042982 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b732f5ff-56d4-4ef8-8261-2ffe5f1cbd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:11:59.117270 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732f5ff-56d4-4ef8-8261-2ffe5f1cbd6e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:11:59.121986 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8e3a6-57ed-47f3-9e9f-5bbb663f929c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:11:59.122870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8e3a6-57ed-47f3-9e9f-5bbb663f929c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:59.123150 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8e3a6-57ed-47f3-9e9f-5bbb663f929c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:59.123455 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8e3a6-57ed-47f3-9e9f-5bbb663f929c admin admin] PATH_INFO: `/v3/users/54e3bdd8f01b4ee5be6ac5e6bc44549b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:59.124682 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8e3a6-57ed-47f3-9e9f-5bbb663f929c admin admin] RBAC: Authorizing `identity:delete_user(user_id=54e3bdd8f01b4ee5be6ac5e6bc44549b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:11:59.126610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8e3a6-57ed-47f3-9e9f-5bbb663f929c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:11:59.290357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8df8e3a6-57ed-47f3-9e9f-5bbb663f929c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54e3bdd8f01b4ee5be6ac5e6bc44549b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:59.290357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8df8e3a6-57ed-47f3-9e9f-5bbb663f929c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54e3bdd8f01b4ee5be6ac5e6bc44549b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:59.311667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8df8e3a6-57ed-47f3-9e9f-5bbb663f929c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54e3bdd8f01b4ee5be6ac5e6bc44549b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:59.312459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8df8e3a6-57ed-47f3-9e9f-5bbb663f929c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54e3bdd8f01b4ee5be6ac5e6bc44549b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:59.326637 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8df8e3a6-57ed-47f3-9e9f-5bbb663f929c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54e3bdd8f01b4ee5be6ac5e6bc44549b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:59.395468 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1183/2329] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:11:59 2022] DELETE /identity/v3/users/54e3bdd8f01b4ee5be6ac5e6bc44549b => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:11:59.629786 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdc3c5-87b9-4f60-9130-1251c39e78f3 service swift] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:11:59.643987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdc3c5-87b9-4f60-9130-1251c39e78f3 service swift] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:11:59.724675 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f454b8-1bfa-4087-8f81-47b8039012d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:11:59.725560 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f454b8-1bfa-4087-8f81-47b8039012d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:59.725849 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f454b8-1bfa-4087-8f81-47b8039012d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:59.726131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f454b8-1bfa-4087-8f81-47b8039012d1 admin admin] PATH_INFO: `/v3/projects/233dcf4525d540b581f144acee94c18d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:59.727394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f454b8-1bfa-4087-8f81-47b8039012d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=233dcf4525d540b581f144acee94c18d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:11:59.729363 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f454b8-1bfa-4087-8f81-47b8039012d1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:11:59.820992 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-86f454b8-1bfa-4087-8f81-47b8039012d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '233dcf4525d540b581f144acee94c18d', 'request_id': 'req-86f454b8-1bfa-4087-8f81-47b8039012d1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:59.885347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-86f454b8-1bfa-4087-8f81-47b8039012d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '233dcf4525d540b581f144acee94c18d', 'request_id': 'req-86f454b8-1bfa-4087-8f81-47b8039012d1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:59.891752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-86f454b8-1bfa-4087-8f81-47b8039012d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '233dcf4525d540b581f144acee94c18d', 'request_id': 'req-86f454b8-1bfa-4087-8f81-47b8039012d1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:59.904927 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-86f454b8-1bfa-4087-8f81-47b8039012d1 admin admin] The token cache is being invalidate because project 233dcf4525d540b581f144acee94c18d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:11:59.905145 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-86f454b8-1bfa-4087-8f81-47b8039012d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:59.905957 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-86f454b8-1bfa-4087-8f81-47b8039012d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:11:59.907608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1147/2330] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:11:59 2022] DELETE /identity/v3/projects/233dcf4525d540b581f144acee94c18d => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:11:59.913412 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0d565-5c29-4d87-8d81-9dbaedee06f4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:11:59.913707 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0d565-5c29-4d87-8d81-9dbaedee06f4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:11:59.914317 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0d565-5c29-4d87-8d81-9dbaedee06f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:11:59.935301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6be0d565-5c29-4d87-8d81-9dbaedee06f4 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:11:59.936495 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6be0d565-5c29-4d87-8d81-9dbaedee06f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:00.011897 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1184/2331] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:11:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:12:00.019985 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e0279-1356-4f34-91f6-8543db6917a1 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:00.020280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e0279-1356-4f34-91f6-8543db6917a1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:00.020758 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e0279-1356-4f34-91f6-8543db6917a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:00.050090 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a09e0279-1356-4f34-91f6-8543db6917a1 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:12:00.051985 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a09e0279-1356-4f34-91f6-8543db6917a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:00.060818 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1148/2332] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:12:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:12:00.064858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0d565-5c29-4d87-8d81-9dbaedee06f4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:12:00.075804 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0d565-5c29-4d87-8d81-9dbaedee06f4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:12:00.086857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abf76b-a7d4-4a97-8034-0672e6a94275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:12:00.088082 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abf76b-a7d4-4a97-8034-0672e6a94275 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:00.088599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abf76b-a7d4-4a97-8034-0672e6a94275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:00.089106 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abf76b-a7d4-4a97-8034-0672e6a94275 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:00.090641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abf76b-a7d4-4a97-8034-0672e6a94275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:12:00.091062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abf76b-a7d4-4a97-8034-0672e6a94275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:12:00.094435 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abf76b-a7d4-4a97-8034-0672e6a94275 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:12:00.105933 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:12:00.109698 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1185/2333] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:12:00.118773 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e0279-1356-4f34-91f6-8543db6917a1 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:12:00.135707 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e0279-1356-4f34-91f6-8543db6917a1 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:12:00.144933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f345d6-97b8-4bf9-81d4-f670124523b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:12:00.146496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f345d6-97b8-4bf9-81d4-f670124523b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:00.147872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f345d6-97b8-4bf9-81d4-f670124523b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:00.148355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f345d6-97b8-4bf9-81d4-f670124523b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:00.149852 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f345d6-97b8-4bf9-81d4-f670124523b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:12:00.153533 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f345d6-97b8-4bf9-81d4-f670124523b8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:12:00.262324 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1149/2334] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:12:00 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:12:00.266830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abf76b-a7d4-4a97-8034-0672e6a94275 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:12:00.285163 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abf76b-a7d4-4a97-8034-0672e6a94275 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:12:00.374757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befac032-1b7d-4c51-b961-9166f720b562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:12:00.375504 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befac032-1b7d-4c51-b961-9166f720b562 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:00.375756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befac032-1b7d-4c51-b961-9166f720b562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:00.376038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befac032-1b7d-4c51-b961-9166f720b562 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:00.376770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befac032-1b7d-4c51-b961-9166f720b562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:12:00.378732 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befac032-1b7d-4c51-b961-9166f720b562 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:12:00.471815 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1186/2335] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:12:00 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:12:00.476131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f345d6-97b8-4bf9-81d4-f670124523b8 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:12:00.487918 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f345d6-97b8-4bf9-81d4-f670124523b8 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:12:00.492291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ac3a7-f10e-4950-bfef-9335d1f6688e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:00.492592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ac3a7-f10e-4950-bfef-9335d1f6688e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:00.492874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ac3a7-f10e-4950-bfef-9335d1f6688e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:00.493296 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b4353-67b7-4224-9b63-3700c4b45931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:12:00.494055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b4353-67b7-4224-9b63-3700c4b45931 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:00.494353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b4353-67b7-4224-9b63-3700c4b45931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:00.494688 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b4353-67b7-4224-9b63-3700c4b45931 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:00.495424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b4353-67b7-4224-9b63-3700c4b45931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:12:00.497218 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b4353-67b7-4224-9b63-3700c4b45931 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:12:00.512212 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1150/2336] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:12:00.513086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-742ac3a7-f10e-4950-bfef-9335d1f6688e None None] MFA Rules not processed for user `4ef0fad9a71d43bfbaba9c87dfed41c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:12:00.514284 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b4353-67b7-4224-9b63-3700c4b45931 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:12:00.514437 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-742ac3a7-f10e-4950-bfef-9335d1f6688e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:00.526394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b4353-67b7-4224-9b63-3700c4b45931 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:12:00.531609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3abd8-f7dd-486c-8740-b4938b9efa65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:12:00.532394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3abd8-f7dd-486c-8740-b4938b9efa65 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:00.532832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3abd8-f7dd-486c-8740-b4938b9efa65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:00.533844 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3abd8-f7dd-486c-8740-b4938b9efa65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:00.534187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3abd8-f7dd-486c-8740-b4938b9efa65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:12:00.535903 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3abd8-f7dd-486c-8740-b4938b9efa65 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:12:00.546958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1151/2337] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:12:00.552911 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3abd8-f7dd-486c-8740-b4938b9efa65 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:12:00.563776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3abd8-f7dd-486c-8740-b4938b9efa65 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:12:00.569094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2dfd9-7f08-4bad-b823-98eded104500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:12:00.570018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2dfd9-7f08-4bad-b823-98eded104500 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:00.570306 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2dfd9-7f08-4bad-b823-98eded104500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:00.570587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2dfd9-7f08-4bad-b823-98eded104500 admin admin] PATH_INFO: `/v3/projects/e773138787be4378b170ddf4a637e634/users/fa27beec2b7643c693588f5d389826f2/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:00.583084 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1187/2338] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 14 19:12:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:12:00.587336 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2dfd9-7f08-4bad-b823-98eded104500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e773138787be4378b170ddf4a637e634, user_id=fa27beec2b7643c693588f5d389826f2, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:12:00.590075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2dfd9-7f08-4bad-b823-98eded104500 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:12:00.595833 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ac3a7-f10e-4950-bfef-9335d1f6688e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:12:00.597783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-742ac3a7-f10e-4950-bfef-9335d1f6688e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:00.657022 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1152/2339] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:12:00 2022] PUT /identity/v3/projects/e773138787be4378b170ddf4a637e634/users/fa27beec2b7643c693588f5d389826f2/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:12:00.671002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff4862-8b3b-4e17-ba30-2da5e58c0286 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:00.671326 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff4862-8b3b-4e17-ba30-2da5e58c0286 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:00.671608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff4862-8b3b-4e17-ba30-2da5e58c0286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:00.679431 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:12:00.681220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ac3a7-f10e-4950-bfef-9335d1f6688e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:12:00.702887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b0ff4862-8b3b-4e17-ba30-2da5e58c0286 None None] MFA Rules not processed for user `fa27beec2b7643c693588f5d389826f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:12:00.704242 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b0ff4862-8b3b-4e17-ba30-2da5e58c0286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:00.716665 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:12:00.744064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7df730-c5db-47c1-b4b8-9efb8896ff0f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4368f68724f946eca53084a9ec7ff590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:12:00.744653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7df730-c5db-47c1-b4b8-9efb8896ff0f service swift] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:00.744950 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7df730-c5db-47c1-b4b8-9efb8896ff0f service swift] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:00.745226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7df730-c5db-47c1-b4b8-9efb8896ff0f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:00.771527 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1153/2340] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:12:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:12:00.783124 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff4862-8b3b-4e17-ba30-2da5e58c0286 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:12:00.785054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b0ff4862-8b3b-4e17-ba30-2da5e58c0286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:00.806289 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7df730-c5db-47c1-b4b8-9efb8896ff0f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:12:00.807950 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7df730-c5db-47c1-b4b8-9efb8896ff0f service swift] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:12:00.817005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1188/2341] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 14 19:12:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 19:12:00.848877 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff4862-8b3b-4e17-ba30-2da5e58c0286 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:12:00.852652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314130c8-5d8d-4e25-b340-fc9c97f96ae4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:12:00.853293 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314130c8-5d8d-4e25-b340-fc9c97f96ae4 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:00.853577 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314130c8-5d8d-4e25-b340-fc9c97f96ae4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:00.853863 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314130c8-5d8d-4e25-b340-fc9c97f96ae4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:00.863022 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314130c8-5d8d-4e25-b340-fc9c97f96ae4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:12:00.864503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314130c8-5d8d-4e25-b340-fc9c97f96ae4 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:12:00.874108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1154/2342] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:12:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:12:06.802766 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cb627-2d02-4bef-87b0-75e4bfba8dd5 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:06.803390 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cb627-2d02-4bef-87b0-75e4bfba8dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:06.803390 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cb627-2d02-4bef-87b0-75e4bfba8dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:06.821150 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-1e3cb627-2d02-4bef-87b0-75e4bfba8dd5 None None] MFA Rules not processed for user `fa27beec2b7643c693588f5d389826f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:12:06.822360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1e3cb627-2d02-4bef-87b0-75e4bfba8dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:06.829927 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1189/2343] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:12:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:12:10.870354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=57 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:12:16.817886 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=40 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:12:24.166602 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cb8de-20b3-4734-9349-a3ad77a6f234 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:24.167130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cb8de-20b3-4734-9349-a3ad77a6f234 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:24.167130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cb8de-20b3-4734-9349-a3ad77a6f234 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:24.168495 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1155/2344] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 14 19:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:12:24.174864 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c8bee-778b-42ba-80f1-7fe46f6b889e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:24.175520 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c8bee-778b-42ba-80f1-7fe46f6b889e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:24.176008 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c8bee-778b-42ba-80f1-7fe46f6b889e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:24.196964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e33c8bee-778b-42ba-80f1-7fe46f6b889e None None] MFA Rules not processed for user `8d27d6df5cb64697adebd961dfbec324`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:12:24.198385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e33c8bee-778b-42ba-80f1-7fe46f6b889e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:24.275538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1190/2345] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 14 19:12:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:12:24.287688 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841cb8de-20b3-4734-9349-a3ad77a6f234 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:12:24.290404 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-841cb8de-20b3-4734-9349-a3ad77a6f234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:24.352367 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841cb8de-20b3-4734-9349-a3ad77a6f234 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:12:24.355973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5a5a7-646d-4d40-9dd7-288b11428ade service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5d76eae62b54521a95f36284118fada', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:12:24.356647 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5a5a7-646d-4d40-9dd7-288b11428ade service glance] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:24.356931 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5a5a7-646d-4d40-9dd7-288b11428ade service glance] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:24.357209 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5a5a7-646d-4d40-9dd7-288b11428ade service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:24.368251 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5a5a7-646d-4d40-9dd7-288b11428ade service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:12:24.369822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5a5a7-646d-4d40-9dd7-288b11428ade service glance] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:12:24.380736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1156/2346] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:12:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:12:24.619088 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec29b7f-add0-4e12-9e91-ef1af95939f8 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:24.619423 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec29b7f-add0-4e12-9e91-ef1af95939f8 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:24.619709 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec29b7f-add0-4e12-9e91-ef1af95939f8 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:24.621245 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1191/2347] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 14 19:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:12:24.627889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef72e5c-255a-40a4-a48b-ff62709fdce0 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:24.628228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef72e5c-255a-40a4-a48b-ff62709fdce0 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:24.628563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef72e5c-255a-40a4-a48b-ff62709fdce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:24.647503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0ef72e5c-255a-40a4-a48b-ff62709fdce0 None None] MFA Rules not processed for user `74b8c75b9cce442b8c430fab6a3fea17`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:12:24.654355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0ef72e5c-255a-40a4-a48b-ff62709fdce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:24.704937 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1157/2348] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 14 19:12:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:12:24.717170 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec29b7f-add0-4e12-9e91-ef1af95939f8 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:12:24.719728 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4ec29b7f-add0-4e12-9e91-ef1af95939f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:24.735398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec29b7f-add0-4e12-9e91-ef1af95939f8 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:12:24.739281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150ed7d-09b8-4396-9d3a-119df93c84ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:12:24.739956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150ed7d-09b8-4396-9d3a-119df93c84ad service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:24.740246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150ed7d-09b8-4396-9d3a-119df93c84ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:24.740549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150ed7d-09b8-4396-9d3a-119df93c84ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:24.748557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150ed7d-09b8-4396-9d3a-119df93c84ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:12:24.750013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150ed7d-09b8-4396-9d3a-119df93c84ad service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:12:24.764396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1192/2349] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:12:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:12:24.849852 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4cc46-149d-452b-b889-b5fda41e56d4 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:24.850009 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4cc46-149d-452b-b889-b5fda41e56d4 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:24.850189 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4cc46-149d-452b-b889-b5fda41e56d4 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:24.851657 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1158/2350] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 14 19:12:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:12:24.859574 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf152fc-7a32-4e44-8646-b6792d908681 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:24.859876 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf152fc-7a32-4e44-8646-b6792d908681 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:24.860161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf152fc-7a32-4e44-8646-b6792d908681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:24.879084 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-4bf152fc-7a32-4e44-8646-b6792d908681 None None] MFA Rules not processed for user `8d27d6df5cb64697adebd961dfbec324`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:12:24.880283 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4bf152fc-7a32-4e44-8646-b6792d908681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:24.899737 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1193/2351] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 14 19:12:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:12:24.912049 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4cc46-149d-452b-b889-b5fda41e56d4 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:12:24.913718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e7c4cc46-149d-452b-b889-b5fda41e56d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:24.927807 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4cc46-149d-452b-b889-b5fda41e56d4 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:12:24.931337 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64a848-c924-42eb-9f76-4cc1aad2fc73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:12:24.931971 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64a848-c924-42eb-9f76-4cc1aad2fc73 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:24.932257 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64a848-c924-42eb-9f76-4cc1aad2fc73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:24.932530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64a848-c924-42eb-9f76-4cc1aad2fc73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:24.943331 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64a848-c924-42eb-9f76-4cc1aad2fc73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:12:24.944766 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64a848-c924-42eb-9f76-4cc1aad2fc73 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:12:24.956139 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1159/2352] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:12:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:12:32.116467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6bc58-65c8-4441-9610-667febbfbd35 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:32.116946 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6bc58-65c8-4441-9610-667febbfbd35 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:32.117048 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6bc58-65c8-4441-9610-667febbfbd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:32.134815 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0ed6bc58-65c8-4441-9610-667febbfbd35 None None] MFA Rules not processed for user `4ef0fad9a71d43bfbaba9c87dfed41c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:12:32.136044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0ed6bc58-65c8-4441-9610-667febbfbd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:32.143105 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1194/2353] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 14 19:12:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:12:32.155943 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64a848-c924-42eb-9f76-4cc1aad2fc73 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:12:32.167097 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64a848-c924-42eb-9f76-4cc1aad2fc73 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:12:32.172182 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc15bc65-8fd7-41ba-9e5a-f049a20c665f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4368f68724f946eca53084a9ec7ff590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:12:32.172823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15bc65-8fd7-41ba-9e5a-f049a20c665f service swift] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:32.173117 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15bc65-8fd7-41ba-9e5a-f049a20c665f service swift] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:32.173401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15bc65-8fd7-41ba-9e5a-f049a20c665f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:32.182126 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc15bc65-8fd7-41ba-9e5a-f049a20c665f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:12:32.183761 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc15bc65-8fd7-41ba-9e5a-f049a20c665f service swift] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:12:32.192001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1160/2354] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 14 19:12:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 19:12:42.139267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=43 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:12:42.188649 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=61 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:12:49.083204 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6bc58-65c8-4441-9610-667febbfbd35 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:12:49.092448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6bc58-65c8-4441-9610-667febbfbd35 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:12:49.096000 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d23dd-cc7e-43dd-b252-bf048698f189 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:12:49.096644 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d23dd-cc7e-43dd-b252-bf048698f189 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:49.096925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d23dd-cc7e-43dd-b252-bf048698f189 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:49.097202 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d23dd-cc7e-43dd-b252-bf048698f189 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:49.098973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cd0d23dd-cc7e-43dd-b252-bf048698f189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:49.111972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d23dd-cc7e-43dd-b252-bf048698f189 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:12:49.113398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d23dd-cc7e-43dd-b252-bf048698f189 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:12:49.124710 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1195/2355] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:12:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:12:49.507368 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc15bc65-8fd7-41ba-9e5a-f049a20c665f service swift] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:12:49.516476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc15bc65-8fd7-41ba-9e5a-f049a20c665f service swift] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:12:49.520064 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b8549-0789-4c69-a4f3-1fd8911afef7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:12:49.520715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b8549-0789-4c69-a4f3-1fd8911afef7 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:49.520996 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b8549-0789-4c69-a4f3-1fd8911afef7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:49.521282 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b8549-0789-4c69-a4f3-1fd8911afef7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:49.523083 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-850b8549-0789-4c69-a4f3-1fd8911afef7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:49.537267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b8549-0789-4c69-a4f3-1fd8911afef7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:12:49.538695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b8549-0789-4c69-a4f3-1fd8911afef7 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:12:49.549695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1161/2356] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:12:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:12:49.847847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a3580-f59a-41d2-b3ff-9d2d7edb3fdc None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:49.848140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a3580-f59a-41d2-b3ff-9d2d7edb3fdc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:49.848432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a3580-f59a-41d2-b3ff-9d2d7edb3fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:49.866113 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-061a3580-f59a-41d2-b3ff-9d2d7edb3fdc None None] MFA Rules not processed for user `51c0e902dbb24a928b093f17a2cb1eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:12:49.867339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-061a3580-f59a-41d2-b3ff-9d2d7edb3fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:49.920908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1196/2357] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:12:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:12:57.284656 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7426f-8e3b-4efa-bf89-d43126a39531 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:57.285620 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7426f-8e3b-4efa-bf89-d43126a39531 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:57.285620 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7426f-8e3b-4efa-bf89-d43126a39531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:57.304870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-11e7426f-8e3b-4efa-bf89-d43126a39531 None None] MFA Rules not processed for user `4ef0fad9a71d43bfbaba9c87dfed41c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:12:57.306175 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-11e7426f-8e3b-4efa-bf89-d43126a39531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:57.314341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1162/2358] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 14 19:12:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:12:57.348580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a3580-f59a-41d2-b3ff-9d2d7edb3fdc None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:12:57.361039 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a3580-f59a-41d2-b3ff-9d2d7edb3fdc None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:12:57.367682 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9d7cd-f619-4738-812d-6bebaa3f665e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4368f68724f946eca53084a9ec7ff590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:12:57.368375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9d7cd-f619-4738-812d-6bebaa3f665e service swift] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:57.368840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9d7cd-f619-4738-812d-6bebaa3f665e service swift] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:57.369124 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9d7cd-f619-4738-812d-6bebaa3f665e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:57.378829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9d7cd-f619-4738-812d-6bebaa3f665e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:12:57.381149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9d7cd-f619-4738-812d-6bebaa3f665e service swift] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:12:57.391760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1197/2359] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 14 19:12:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 19:12:58.292325 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd040b68-94cf-472f-a2ed-baefa873b339 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:58.292801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd040b68-94cf-472f-a2ed-baefa873b339 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:58.293571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd040b68-94cf-472f-a2ed-baefa873b339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:58.311473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-bd040b68-94cf-472f-a2ed-baefa873b339 None None] MFA Rules not processed for user `4ef0fad9a71d43bfbaba9c87dfed41c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:12:58.312745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-bd040b68-94cf-472f-a2ed-baefa873b339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:58.320055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1163/2360] 10.222.0.69 () {58 vars in 967 bytes} [Thu Jul 14 19:12:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:12:58.332238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9d7cd-f619-4738-812d-6bebaa3f665e service swift] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:12:58.342907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9d7cd-f619-4738-812d-6bebaa3f665e service swift] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:12:58.348874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54b32d-7678-4acb-9999-4413bedb6d57 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4368f68724f946eca53084a9ec7ff590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:12:58.349521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54b32d-7678-4acb-9999-4413bedb6d57 service swift] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:58.349810 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54b32d-7678-4acb-9999-4413bedb6d57 service swift] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:58.350092 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54b32d-7678-4acb-9999-4413bedb6d57 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:58.358633 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54b32d-7678-4acb-9999-4413bedb6d57 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:12:58.360257 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54b32d-7678-4acb-9999-4413bedb6d57 service swift] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:12:58.369138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1198/2361] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 14 19:12:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 19:12:58.975928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2069b1-abb7-4301-8f7e-625c92009620 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:58.976225 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2069b1-abb7-4301-8f7e-625c92009620 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:58.976511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2069b1-abb7-4301-8f7e-625c92009620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:59.001631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-6e2069b1-abb7-4301-8f7e-625c92009620 None None] MFA Rules not processed for user `4ef0fad9a71d43bfbaba9c87dfed41c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:12:59.002913 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6e2069b1-abb7-4301-8f7e-625c92009620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:59.011813 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1164/2362] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 14 19:12:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:12:59.024599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54b32d-7678-4acb-9999-4413bedb6d57 service swift] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:12:59.037300 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54b32d-7678-4acb-9999-4413bedb6d57 service swift] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:12:59.042497 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db623ef3-e845-4def-be90-8792ba220240 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4368f68724f946eca53084a9ec7ff590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:12:59.043305 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db623ef3-e845-4def-be90-8792ba220240 service swift] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:59.043599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db623ef3-e845-4def-be90-8792ba220240 service swift] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:59.043882 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db623ef3-e845-4def-be90-8792ba220240 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:59.052857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db623ef3-e845-4def-be90-8792ba220240 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:12:59.054473 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db623ef3-e845-4def-be90-8792ba220240 service swift] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:12:59.063053 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1199/2363] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 14 19:12:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 19:12:59.689990 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25f175-121f-4735-9313-7d1239dc17b6 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:59.690570 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25f175-121f-4735-9313-7d1239dc17b6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:59.690570 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25f175-121f-4735-9313-7d1239dc17b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:59.709643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-af25f175-121f-4735-9313-7d1239dc17b6 None None] MFA Rules not processed for user `4ef0fad9a71d43bfbaba9c87dfed41c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:12:59.710868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-af25f175-121f-4735-9313-7d1239dc17b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:12:59.720571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1165/2364] 10.222.0.69 () {58 vars in 967 bytes} [Thu Jul 14 19:12:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:12:59.733689 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db623ef3-e845-4def-be90-8792ba220240 service swift] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:12:59.744625 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db623ef3-e845-4def-be90-8792ba220240 service swift] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:12:59.749662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb26e37-c035-412a-93d2-f8237d97b4b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4368f68724f946eca53084a9ec7ff590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:12:59.750719 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb26e37-c035-412a-93d2-f8237d97b4b0 service swift] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:12:59.750719 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb26e37-c035-412a-93d2-f8237d97b4b0 service swift] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:12:59.750922 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb26e37-c035-412a-93d2-f8237d97b4b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:12:59.759399 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb26e37-c035-412a-93d2-f8237d97b4b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:12:59.761012 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb26e37-c035-412a-93d2-f8237d97b4b0 service swift] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:12:59.769531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1200/2365] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 14 19:12:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 19:13:02.331341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25f175-121f-4735-9313-7d1239dc17b6 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:02.392853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25f175-121f-4735-9313-7d1239dc17b6 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:02.397847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c7fc3-251d-490a-b9a0-8f158f4bbd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:02.398740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c7fc3-251d-490a-b9a0-8f158f4bbd06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:02.399002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c7fc3-251d-490a-b9a0-8f158f4bbd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:02.399575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c7fc3-251d-490a-b9a0-8f158f4bbd06 admin admin] PATH_INFO: `/v3/users/fa27beec2b7643c693588f5d389826f2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:02.400761 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c7fc3-251d-490a-b9a0-8f158f4bbd06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa27beec2b7643c693588f5d389826f2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:02.402696 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c7fc3-251d-490a-b9a0-8f158f4bbd06 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:02.538868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-104c7fc3-251d-490a-b9a0-8f158f4bbd06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa27beec2b7643c693588f5d389826f2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:02.546423 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-104c7fc3-251d-490a-b9a0-8f158f4bbd06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa27beec2b7643c693588f5d389826f2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:02.547132 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-104c7fc3-251d-490a-b9a0-8f158f4bbd06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa27beec2b7643c693588f5d389826f2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:02.547764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-104c7fc3-251d-490a-b9a0-8f158f4bbd06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa27beec2b7643c693588f5d389826f2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:02.560219 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-104c7fc3-251d-490a-b9a0-8f158f4bbd06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa27beec2b7643c693588f5d389826f2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:02.606353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1166/2366] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:13:02 2022] DELETE /identity/v3/users/fa27beec2b7643c693588f5d389826f2 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:13:02.610881 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8447d-0ad1-4c8d-b955-b13395a409ca None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:02.611406 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8447d-0ad1-4c8d-b955-b13395a409ca None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:02.611503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8447d-0ad1-4c8d-b955-b13395a409ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:02.629172 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-05e8447d-0ad1-4c8d-b955-b13395a409ca None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:13:02.630426 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-05e8447d-0ad1-4c8d-b955-b13395a409ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:02.689370 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1201/2367] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:13:02.701341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c7fc3-251d-490a-b9a0-8f158f4bbd06 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:02.704159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-104c7fc3-251d-490a-b9a0-8f158f4bbd06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:02.761132 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:13:02.776905 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c7fc3-251d-490a-b9a0-8f158f4bbd06 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:02.780852 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271d614-32b1-4349-abf7-d4353356680e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:02.781513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271d614-32b1-4349-abf7-d4353356680e service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:02.781829 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271d614-32b1-4349-abf7-d4353356680e service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:02.782112 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271d614-32b1-4349-abf7-d4353356680e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:02.791391 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271d614-32b1-4349-abf7-d4353356680e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:02.792966 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271d614-32b1-4349-abf7-d4353356680e service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:02.802672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1167/2368] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:13:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:13:03.075927 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8447d-0ad1-4c8d-b955-b13395a409ca None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:03.077456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-05e8447d-0ad1-4c8d-b955-b13395a409ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:03.089669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8447d-0ad1-4c8d-b955-b13395a409ca None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:03.095558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859052b4-7c79-4491-bd03-4872c01ddae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:03.096639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859052b4-7c79-4491-bd03-4872c01ddae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:03.096920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859052b4-7c79-4491-bd03-4872c01ddae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:03.097294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859052b4-7c79-4491-bd03-4872c01ddae0 admin admin] PATH_INFO: `/v3/projects/e773138787be4378b170ddf4a637e634` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:03.098695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859052b4-7c79-4491-bd03-4872c01ddae0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e773138787be4378b170ddf4a637e634)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:03.101485 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859052b4-7c79-4491-bd03-4872c01ddae0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:03.203662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:13:03.213718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-859052b4-7c79-4491-bd03-4872c01ddae0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e773138787be4378b170ddf4a637e634', 'request_id': 'req-859052b4-7c79-4491-bd03-4872c01ddae0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:03.333304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-859052b4-7c79-4491-bd03-4872c01ddae0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e773138787be4378b170ddf4a637e634', 'request_id': 'req-859052b4-7c79-4491-bd03-4872c01ddae0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:03.343712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-859052b4-7c79-4491-bd03-4872c01ddae0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e773138787be4378b170ddf4a637e634', 'request_id': 'req-859052b4-7c79-4491-bd03-4872c01ddae0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:03.355727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-859052b4-7c79-4491-bd03-4872c01ddae0 admin admin] The token cache is being invalidate because project e773138787be4378b170ddf4a637e634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:13:03.356026 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-859052b4-7c79-4491-bd03-4872c01ddae0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:03.356910 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-859052b4-7c79-4491-bd03-4872c01ddae0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:03.360191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1202/2369] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:13:03 2022] DELETE /identity/v3/projects/e773138787be4378b170ddf4a637e634 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:13:03.365810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e37de-fde9-42f4-879b-1d2658ed48ac None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:03.366341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e37de-fde9-42f4-879b-1d2658ed48ac None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:03.366454 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e37de-fde9-42f4-879b-1d2658ed48ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:03.401632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0b0e37de-fde9-42f4-879b-1d2658ed48ac None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:13:03.402908 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0b0e37de-fde9-42f4-879b-1d2658ed48ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:03.478588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1168/2370] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:13:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:13:03.486638 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982926c-0a34-49ca-89bf-5b359b4ade9d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:03.486936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982926c-0a34-49ca-89bf-5b359b4ade9d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:03.489429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982926c-0a34-49ca-89bf-5b359b4ade9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:03.509503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2982926c-0a34-49ca-89bf-5b359b4ade9d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:13:03.510451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2982926c-0a34-49ca-89bf-5b359b4ade9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:03.523193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1203/2371] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:13:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:13:03.524581 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e37de-fde9-42f4-879b-1d2658ed48ac None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:03.534805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e37de-fde9-42f4-879b-1d2658ed48ac None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:03.539626 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7695e0-387e-4956-95bb-f3105720d5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:03.540381 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7695e0-387e-4956-95bb-f3105720d5b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:03.540745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7695e0-387e-4956-95bb-f3105720d5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:03.541097 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7695e0-387e-4956-95bb-f3105720d5b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:03.541913 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7695e0-387e-4956-95bb-f3105720d5b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:13:03.542320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7695e0-387e-4956-95bb-f3105720d5b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:03.544092 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7695e0-387e-4956-95bb-f3105720d5b9 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:03.560666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1169/2372] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:13:03.562262 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982926c-0a34-49ca-89bf-5b359b4ade9d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:03.572381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982926c-0a34-49ca-89bf-5b359b4ade9d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:03.577770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca789595-bdf4-4a5c-bfaf-ac1d6d61baa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:03.578419 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca789595-bdf4-4a5c-bfaf-ac1d6d61baa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:03.578708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca789595-bdf4-4a5c-bfaf-ac1d6d61baa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:03.578990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca789595-bdf4-4a5c-bfaf-ac1d6d61baa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:03.579734 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca789595-bdf4-4a5c-bfaf-ac1d6d61baa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:03.581655 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca789595-bdf4-4a5c-bfaf-ac1d6d61baa2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:03.652493 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1204/2373] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:13:03 2022] POST /identity/v3/projects => generated 391 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:13:03.656654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7695e0-387e-4956-95bb-f3105720d5b9 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:03.667476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7695e0-387e-4956-95bb-f3105720d5b9 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:03.724738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3c283-ac2e-4077-988e-f1470c86d5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:03.725405 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3c283-ac2e-4077-988e-f1470c86d5f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:03.725691 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3c283-ac2e-4077-988e-f1470c86d5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:03.725967 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3c283-ac2e-4077-988e-f1470c86d5f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:03.726667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3c283-ac2e-4077-988e-f1470c86d5f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:03.728677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3c283-ac2e-4077-988e-f1470c86d5f6 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:03.793026 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1170/2374] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:13:03 2022] POST /identity/v3/users => generated 296 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:13:03.797159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca789595-bdf4-4a5c-bfaf-ac1d6d61baa2 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:03.808407 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca789595-bdf4-4a5c-bfaf-ac1d6d61baa2 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:03.813570 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359e05c-86d2-4020-86fa-a96862efa7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:03.814247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359e05c-86d2-4020-86fa-a96862efa7ca admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:03.814537 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359e05c-86d2-4020-86fa-a96862efa7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:03.814820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359e05c-86d2-4020-86fa-a96862efa7ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:03.815589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359e05c-86d2-4020-86fa-a96862efa7ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:03.817201 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359e05c-86d2-4020-86fa-a96862efa7ca admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:03.828336 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1205/2375] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:13:03.833005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3c283-ac2e-4077-988e-f1470c86d5f6 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:03.845104 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3c283-ac2e-4077-988e-f1470c86d5f6 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:03.850682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8cc37-eca8-48a8-9b25-0056b964dd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:03.851417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8cc37-eca8-48a8-9b25-0056b964dd6e admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:03.851729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8cc37-eca8-48a8-9b25-0056b964dd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:03.852009 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8cc37-eca8-48a8-9b25-0056b964dd6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:03.852764 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8cc37-eca8-48a8-9b25-0056b964dd6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:03.854369 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8cc37-eca8-48a8-9b25-0056b964dd6e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:03.865128 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1171/2376] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:13:03.869349 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359e05c-86d2-4020-86fa-a96862efa7ca admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:03.879701 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359e05c-86d2-4020-86fa-a96862efa7ca admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:03.884432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8bc334-f944-4824-aff6-cf306dd973c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:03.885372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bc334-f944-4824-aff6-cf306dd973c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:03.885960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bc334-f944-4824-aff6-cf306dd973c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:03.885960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bc334-f944-4824-aff6-cf306dd973c4 admin admin] PATH_INFO: `/v3/projects/ac45ff8a820a43b3981ef19d7ea6567e/users/abb0bf6df3c1439081b4088e400c7b97/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:03.903681 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8bc334-f944-4824-aff6-cf306dd973c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac45ff8a820a43b3981ef19d7ea6567e, user_id=abb0bf6df3c1439081b4088e400c7b97, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:03.906369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8bc334-f944-4824-aff6-cf306dd973c4 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:03.965926 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1206/2377] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:13:03 2022] PUT /identity/v3/projects/ac45ff8a820a43b3981ef19d7ea6567e/users/abb0bf6df3c1439081b4088e400c7b97/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:13:03.977035 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ce983-b85c-4073-b6df-b328f0b0e939 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:03.977382 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ce983-b85c-4073-b6df-b328f0b0e939 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:03.977667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ce983-b85c-4073-b6df-b328f0b0e939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:04.009384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a58ce983-b85c-4073-b6df-b328f0b0e939 None None] MFA Rules not processed for user `abb0bf6df3c1439081b4088e400c7b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:13:04.010699 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a58ce983-b85c-4073-b6df-b328f0b0e939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:04.054125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:13:04.082327 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1172/2378] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:13:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:13:04.093832 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8bc334-f944-4824-aff6-cf306dd973c4 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:04.095757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ac8bc334-f944-4824-aff6-cf306dd973c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:04.159368 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8bc334-f944-4824-aff6-cf306dd973c4 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:04.163241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72406b45-33c3-4bb8-9ff4-3d72fc1ceb10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:04.163928 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72406b45-33c3-4bb8-9ff4-3d72fc1ceb10 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:04.164219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72406b45-33c3-4bb8-9ff4-3d72fc1ceb10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:04.164503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72406b45-33c3-4bb8-9ff4-3d72fc1ceb10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:04.172840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72406b45-33c3-4bb8-9ff4-3d72fc1ceb10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:04.174290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72406b45-33c3-4bb8-9ff4-3d72fc1ceb10 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:04.184480 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1207/2379] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:13:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:13:05.733312 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2440c-6ace-41b5-9e53-51ef6432f446 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:05.733819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2440c-6ace-41b5-9e53-51ef6432f446 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:05.733922 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2440c-6ace-41b5-9e53-51ef6432f446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:05.752700 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0bb2440c-6ace-41b5-9e53-51ef6432f446 None None] MFA Rules not processed for user `4ef0fad9a71d43bfbaba9c87dfed41c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:13:05.754844 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0bb2440c-6ace-41b5-9e53-51ef6432f446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:05.820676 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1173/2380] 10.222.0.69 () {58 vars in 967 bytes} [Thu Jul 14 19:13:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:13:05.836735 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72406b45-33c3-4bb8-9ff4-3d72fc1ceb10 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:05.838875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-72406b45-33c3-4bb8-9ff4-3d72fc1ceb10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:05.915317 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:13:05.920000 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72406b45-33c3-4bb8-9ff4-3d72fc1ceb10 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:05.925082 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00524fd4-ed39-4333-bbf1-eba921de32b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4368f68724f946eca53084a9ec7ff590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:05.925763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00524fd4-ed39-4333-bbf1-eba921de32b0 service swift] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:05.926644 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00524fd4-ed39-4333-bbf1-eba921de32b0 service swift] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:05.926927 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00524fd4-ed39-4333-bbf1-eba921de32b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:05.935214 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00524fd4-ed39-4333-bbf1-eba921de32b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:05.936817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00524fd4-ed39-4333-bbf1-eba921de32b0 service swift] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:05.944906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1208/2381] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 14 19:13:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 19:13:15.812203 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=35 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:13:16.095421 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:13:34.006681 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2440c-6ace-41b5-9e53-51ef6432f446 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:34.070688 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2440c-6ace-41b5-9e53-51ef6432f446 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:34.075412 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168b34b-c35d-4970-a7ea-a3dcfa4fbc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:34.076281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168b34b-c35d-4970-a7ea-a3dcfa4fbc08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:34.076564 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168b34b-c35d-4970-a7ea-a3dcfa4fbc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:34.076865 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168b34b-c35d-4970-a7ea-a3dcfa4fbc08 admin admin] PATH_INFO: `/v3/users/abb0bf6df3c1439081b4088e400c7b97` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:34.078048 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168b34b-c35d-4970-a7ea-a3dcfa4fbc08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abb0bf6df3c1439081b4088e400c7b97)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:34.080022 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168b34b-c35d-4970-a7ea-a3dcfa4fbc08 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:34.312166 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c168b34b-c35d-4970-a7ea-a3dcfa4fbc08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abb0bf6df3c1439081b4088e400c7b97'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:34.325953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c168b34b-c35d-4970-a7ea-a3dcfa4fbc08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abb0bf6df3c1439081b4088e400c7b97'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:34.326715 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c168b34b-c35d-4970-a7ea-a3dcfa4fbc08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abb0bf6df3c1439081b4088e400c7b97'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:34.327391 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c168b34b-c35d-4970-a7ea-a3dcfa4fbc08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abb0bf6df3c1439081b4088e400c7b97'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:34.339738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c168b34b-c35d-4970-a7ea-a3dcfa4fbc08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abb0bf6df3c1439081b4088e400c7b97'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:34.410165 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1174/2382] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:13:34 2022] DELETE /identity/v3/users/abb0bf6df3c1439081b4088e400c7b97 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:13:34.415354 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454da2dc-87e0-4961-bfa8-58f4ee67f2c2 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:34.415825 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454da2dc-87e0-4961-bfa8-58f4ee67f2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:34.415938 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454da2dc-87e0-4961-bfa8-58f4ee67f2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:34.433798 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-454da2dc-87e0-4961-bfa8-58f4ee67f2c2 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:13:34.435016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-454da2dc-87e0-4961-bfa8-58f4ee67f2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:34.500563 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1209/2383] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:13:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:13:34.510274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168b34b-c35d-4970-a7ea-a3dcfa4fbc08 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:34.511778 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c168b34b-c35d-4970-a7ea-a3dcfa4fbc08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:34.569774 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168b34b-c35d-4970-a7ea-a3dcfa4fbc08 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:34.573302 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad23c3-3689-4a7a-9f25-a778958bd079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:34.573953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad23c3-3689-4a7a-9f25-a778958bd079 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:34.574244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad23c3-3689-4a7a-9f25-a778958bd079 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:34.574525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad23c3-3689-4a7a-9f25-a778958bd079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:34.584504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad23c3-3689-4a7a-9f25-a778958bd079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:34.585956 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad23c3-3689-4a7a-9f25-a778958bd079 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:34.595603 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1175/2384] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:13:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:13:34.989003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454da2dc-87e0-4961-bfa8-58f4ee67f2c2 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:34.991789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-454da2dc-87e0-4961-bfa8-58f4ee67f2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:35.005107 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454da2dc-87e0-4961-bfa8-58f4ee67f2c2 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:35.011101 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e1d59-aa06-4496-9d58-b00d09f03d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:35.012050 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e1d59-aa06-4496-9d58-b00d09f03d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:35.012339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e1d59-aa06-4496-9d58-b00d09f03d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:35.012621 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e1d59-aa06-4496-9d58-b00d09f03d7a admin admin] PATH_INFO: `/v3/projects/ac45ff8a820a43b3981ef19d7ea6567e` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:35.014060 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e1d59-aa06-4496-9d58-b00d09f03d7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac45ff8a820a43b3981ef19d7ea6567e)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:35.016229 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e1d59-aa06-4496-9d58-b00d09f03d7a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:35.101908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-095e1d59-aa06-4496-9d58-b00d09f03d7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac45ff8a820a43b3981ef19d7ea6567e', 'request_id': 'req-095e1d59-aa06-4496-9d58-b00d09f03d7a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:35.181553 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-095e1d59-aa06-4496-9d58-b00d09f03d7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac45ff8a820a43b3981ef19d7ea6567e', 'request_id': 'req-095e1d59-aa06-4496-9d58-b00d09f03d7a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:35.188522 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-095e1d59-aa06-4496-9d58-b00d09f03d7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac45ff8a820a43b3981ef19d7ea6567e', 'request_id': 'req-095e1d59-aa06-4496-9d58-b00d09f03d7a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:35.200310 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-095e1d59-aa06-4496-9d58-b00d09f03d7a admin admin] The token cache is being invalidate because project ac45ff8a820a43b3981ef19d7ea6567e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:13:35.200602 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-095e1d59-aa06-4496-9d58-b00d09f03d7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:35.201246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-095e1d59-aa06-4496-9d58-b00d09f03d7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:13:35.202811 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1210/2385] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:13:34 2022] DELETE /identity/v3/projects/ac45ff8a820a43b3981ef19d7ea6567e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:13:35.207828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c1fe0-3e9a-411a-98ea-e302d0859c9d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:35.208357 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c1fe0-3e9a-411a-98ea-e302d0859c9d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:35.209076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c1fe0-3e9a-411a-98ea-e302d0859c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:35.227686 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-988c1fe0-3e9a-411a-98ea-e302d0859c9d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:13:35.228899 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-988c1fe0-3e9a-411a-98ea-e302d0859c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:35.246091 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:13:35.289921 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1176/2386] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:13:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:13:35.297878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698ed5e-8b9b-407c-8cad-ff6f0d78447c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:35.298169 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698ed5e-8b9b-407c-8cad-ff6f0d78447c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:35.298448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698ed5e-8b9b-407c-8cad-ff6f0d78447c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:35.317375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-1698ed5e-8b9b-407c-8cad-ff6f0d78447c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:13:35.319230 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1698ed5e-8b9b-407c-8cad-ff6f0d78447c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:35.327131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1211/2387] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:13:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:13:35.331397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c1fe0-3e9a-411a-98ea-e302d0859c9d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:35.342406 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c1fe0-3e9a-411a-98ea-e302d0859c9d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:35.347446 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba6cca-e8af-490c-be49-c1d6f01499ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:35.348124 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba6cca-e8af-490c-be49-c1d6f01499ae admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:35.348413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba6cca-e8af-490c-be49-c1d6f01499ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:35.348690 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba6cca-e8af-490c-be49-c1d6f01499ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:35.349534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba6cca-e8af-490c-be49-c1d6f01499ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:13:35.349848 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba6cca-e8af-490c-be49-c1d6f01499ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:35.351502 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba6cca-e8af-490c-be49-c1d6f01499ae admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:35.365611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1177/2388] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:13:35.369951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698ed5e-8b9b-407c-8cad-ff6f0d78447c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:35.379438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698ed5e-8b9b-407c-8cad-ff6f0d78447c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:35.387212 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251fea8-2a48-4df2-99d8-248a1d4750fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:35.388154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251fea8-2a48-4df2-99d8-248a1d4750fc admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:35.388441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251fea8-2a48-4df2-99d8-248a1d4750fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:35.388718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251fea8-2a48-4df2-99d8-248a1d4750fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:35.389445 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251fea8-2a48-4df2-99d8-248a1d4750fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:35.391392 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251fea8-2a48-4df2-99d8-248a1d4750fc admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:35.468262 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1212/2389] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:13:35 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:13:35.472566 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba6cca-e8af-490c-be49-c1d6f01499ae admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:35.483595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba6cca-e8af-490c-be49-c1d6f01499ae admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:35.674843 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7794a2e-2bf2-457e-a82d-91ed5ce200f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:35.675448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7794a2e-2bf2-457e-a82d-91ed5ce200f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:35.675735 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7794a2e-2bf2-457e-a82d-91ed5ce200f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:35.676010 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7794a2e-2bf2-457e-a82d-91ed5ce200f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:35.676706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7794a2e-2bf2-457e-a82d-91ed5ce200f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:35.679288 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7794a2e-2bf2-457e-a82d-91ed5ce200f1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:35.793712 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1178/2390] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:13:35 2022] POST /identity/v3/users => generated 290 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:13:35.797858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251fea8-2a48-4df2-99d8-248a1d4750fc admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:35.809097 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251fea8-2a48-4df2-99d8-248a1d4750fc admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:35.813824 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eeb179-c375-44ba-baf4-eb01304348a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:35.814488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eeb179-c375-44ba-baf4-eb01304348a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:35.818386 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eeb179-c375-44ba-baf4-eb01304348a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:35.818386 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eeb179-c375-44ba-baf4-eb01304348a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:35.818386 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eeb179-c375-44ba-baf4-eb01304348a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:35.818386 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eeb179-c375-44ba-baf4-eb01304348a7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:35.829222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1213/2391] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:13:35.833230 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7794a2e-2bf2-457e-a82d-91ed5ce200f1 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:35.843615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7794a2e-2bf2-457e-a82d-91ed5ce200f1 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:35.848624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5931d09f-8b73-4985-9349-045b41732591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:35.849970 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931d09f-8b73-4985-9349-045b41732591 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:35.850308 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931d09f-8b73-4985-9349-045b41732591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:35.850587 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931d09f-8b73-4985-9349-045b41732591 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:35.851360 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5931d09f-8b73-4985-9349-045b41732591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:35.858114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5931d09f-8b73-4985-9349-045b41732591 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:35.864126 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1179/2392] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:13:35.868990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eeb179-c375-44ba-baf4-eb01304348a7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:35.879277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eeb179-c375-44ba-baf4-eb01304348a7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:35.883813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615808b-77aa-4e14-af94-47768f5fb226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:35.884699 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615808b-77aa-4e14-af94-47768f5fb226 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:35.885192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615808b-77aa-4e14-af94-47768f5fb226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:35.885481 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615808b-77aa-4e14-af94-47768f5fb226 admin admin] PATH_INFO: `/v3/projects/329192399b054c01a7f28f4c6fb4d484/users/89bf838da11b41bc9b9f77c9f10a6bbd/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:35.900680 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615808b-77aa-4e14-af94-47768f5fb226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=329192399b054c01a7f28f4c6fb4d484, user_id=89bf838da11b41bc9b9f77c9f10a6bbd, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:35.903383 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615808b-77aa-4e14-af94-47768f5fb226 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:36.000631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1214/2393] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:13:35 2022] PUT /identity/v3/projects/329192399b054c01a7f28f4c6fb4d484/users/89bf838da11b41bc9b9f77c9f10a6bbd/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:13:36.005591 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af759fa0-16e9-491a-9024-ac0f48801387 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:36.005912 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af759fa0-16e9-491a-9024-ac0f48801387 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:36.006196 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af759fa0-16e9-491a-9024-ac0f48801387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:36.024248 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-af759fa0-16e9-491a-9024-ac0f48801387 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:13:36.025723 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-af759fa0-16e9-491a-9024-ac0f48801387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:36.083125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1180/2394] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:13:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:13:36.093186 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615808b-77aa-4e14-af94-47768f5fb226 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:36.094858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2615808b-77aa-4e14-af94-47768f5fb226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:36.117266 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:13:36.153303 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615808b-77aa-4e14-af94-47768f5fb226 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:36.157172 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a4565-e654-43ac-885a-68a0874bfcee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:36.158077 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a4565-e654-43ac-885a-68a0874bfcee service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:36.158561 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a4565-e654-43ac-885a-68a0874bfcee service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:36.159219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a4565-e654-43ac-885a-68a0874bfcee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:36.169188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a4565-e654-43ac-885a-68a0874bfcee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:36.171025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a4565-e654-43ac-885a-68a0874bfcee service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:36.181169 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1215/2395] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:13:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:13:41.513993 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9489b9-c54a-48a1-9ac0-00a1cbcd4906 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:41.514453 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9489b9-c54a-48a1-9ac0-00a1cbcd4906 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:41.514564 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9489b9-c54a-48a1-9ac0-00a1cbcd4906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:41.549911 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-5e9489b9-c54a-48a1-9ac0-00a1cbcd4906 None None] MFA Rules not processed for user `89bf838da11b41bc9b9f77c9f10a6bbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:13:41.551260 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5e9489b9-c54a-48a1-9ac0-00a1cbcd4906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:41.570382 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:13:41.637099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1181/2396] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:13:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:13:41.645946 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a4565-e654-43ac-885a-68a0874bfcee service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:41.657531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a4565-e654-43ac-885a-68a0874bfcee service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:41.664355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f75b7e-bfd1-4e39-8350-e8f5d1e9147c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:41.665034 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f75b7e-bfd1-4e39-8350-e8f5d1e9147c admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:41.665321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f75b7e-bfd1-4e39-8350-e8f5d1e9147c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:41.665598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f75b7e-bfd1-4e39-8350-e8f5d1e9147c admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:41.666391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f75b7e-bfd1-4e39-8350-e8f5d1e9147c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:41.668521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f75b7e-bfd1-4e39-8350-e8f5d1e9147c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:41.762513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1216/2397] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:13:41 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:13:41.766550 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9489b9-c54a-48a1-9ac0-00a1cbcd4906 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:41.779205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9489b9-c54a-48a1-9ac0-00a1cbcd4906 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:41.846561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d788b57-c477-4857-8a3a-5a2276463ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:41.847293 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d788b57-c477-4857-8a3a-5a2276463ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:41.847647 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d788b57-c477-4857-8a3a-5a2276463ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:41.847869 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d788b57-c477-4857-8a3a-5a2276463ea4 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:41.848647 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d788b57-c477-4857-8a3a-5a2276463ea4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:41.850592 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d788b57-c477-4857-8a3a-5a2276463ea4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:41.961045 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1182/2398] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:13:41 2022] POST /identity/v3/users => generated 296 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:13:41.965178 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f75b7e-bfd1-4e39-8350-e8f5d1e9147c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:41.976824 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f75b7e-bfd1-4e39-8350-e8f5d1e9147c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:41.981728 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92092a-c5ba-4016-9333-2e99a4c6acaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:41.982400 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92092a-c5ba-4016-9333-2e99a4c6acaa admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:41.982689 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92092a-c5ba-4016-9333-2e99a4c6acaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:41.982971 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92092a-c5ba-4016-9333-2e99a4c6acaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:41.983776 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92092a-c5ba-4016-9333-2e99a4c6acaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:41.985412 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92092a-c5ba-4016-9333-2e99a4c6acaa admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:41.998135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1217/2399] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:13:42.004245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d788b57-c477-4857-8a3a-5a2276463ea4 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:42.014747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d788b57-c477-4857-8a3a-5a2276463ea4 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:42.019385 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e2245-d73e-4bec-a913-4bb1c14bffa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:42.020376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e2245-d73e-4bec-a913-4bb1c14bffa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:42.026004 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e2245-d73e-4bec-a913-4bb1c14bffa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:42.026004 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e2245-d73e-4bec-a913-4bb1c14bffa5 admin admin] PATH_INFO: `/v3/projects/741d8a8b02c949dab983ed7f56f6c2ac/users/a869e695cc9045988ecbf179be72d459/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:42.037667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e2245-d73e-4bec-a913-4bb1c14bffa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=741d8a8b02c949dab983ed7f56f6c2ac, user_id=a869e695cc9045988ecbf179be72d459, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:42.040424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e2245-d73e-4bec-a913-4bb1c14bffa5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:42.136388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1183/2400] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:13:42 2022] PUT /identity/v3/projects/741d8a8b02c949dab983ed7f56f6c2ac/users/a869e695cc9045988ecbf179be72d459/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:13:42.138541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92092a-c5ba-4016-9333-2e99a4c6acaa admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:42.158006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92092a-c5ba-4016-9333-2e99a4c6acaa admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:42.230573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbef0c5-e225-42eb-b293-09fe5b1afb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:42.231338 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbef0c5-e225-42eb-b293-09fe5b1afb55 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:42.232173 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbef0c5-e225-42eb-b293-09fe5b1afb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:42.232467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbef0c5-e225-42eb-b293-09fe5b1afb55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:42.233209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbef0c5-e225-42eb-b293-09fe5b1afb55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:42.234924 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbef0c5-e225-42eb-b293-09fe5b1afb55 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:42.247984 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1218/2401] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:13:42.252476 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e2245-d73e-4bec-a913-4bb1c14bffa5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:42.264529 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e2245-d73e-4bec-a913-4bb1c14bffa5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:42.270616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b90b0-e3f0-4f4c-94c4-772f4dbccb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:42.272090 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b90b0-e3f0-4f4c-94c4-772f4dbccb9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:42.272477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b90b0-e3f0-4f4c-94c4-772f4dbccb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:42.272899 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b90b0-e3f0-4f4c-94c4-772f4dbccb9c admin admin] PATH_INFO: `/v3/projects/741d8a8b02c949dab983ed7f56f6c2ac/users/a869e695cc9045988ecbf179be72d459/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:42.275492 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b90b0-e3f0-4f4c-94c4-772f4dbccb9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=741d8a8b02c949dab983ed7f56f6c2ac, user_id=a869e695cc9045988ecbf179be72d459, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:42.278336 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b90b0-e3f0-4f4c-94c4-772f4dbccb9c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:42.290691 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1184/2402] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:13:42 2022] PUT /identity/v3/projects/741d8a8b02c949dab983ed7f56f6c2ac/users/a869e695cc9045988ecbf179be72d459/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:13:46.064780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbef0c5-e225-42eb-b293-09fe5b1afb55 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:46.066774 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ccbef0c5-e225-42eb-b293-09fe5b1afb55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:46.148339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbef0c5-e225-42eb-b293-09fe5b1afb55 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:46.203299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:13:46.270035 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d468106-9fc5-4c0b-a97d-e1fdba74d13b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:46.270865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d468106-9fc5-4c0b-a97d-e1fdba74d13b service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:46.271392 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d468106-9fc5-4c0b-a97d-e1fdba74d13b service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:46.272107 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d468106-9fc5-4c0b-a97d-e1fdba74d13b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:46.274069 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4d468106-9fc5-4c0b-a97d-e1fdba74d13b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:46.349775 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d468106-9fc5-4c0b-a97d-e1fdba74d13b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:46.351581 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d468106-9fc5-4c0b-a97d-e1fdba74d13b service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:46.366484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1219/2403] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:13:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:13:48.383471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a79e96-3371-44c0-861a-900b338e3b95 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:48.383984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a79e96-3371-44c0-861a-900b338e3b95 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:48.384347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a79e96-3371-44c0-861a-900b338e3b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:48.443455 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-92a79e96-3371-44c0-861a-900b338e3b95 None None] MFA Rules not processed for user `a869e695cc9045988ecbf179be72d459`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:13:48.444794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-92a79e96-3371-44c0-861a-900b338e3b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:48.519856 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:13:48.530807 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1185/2404] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:13:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:13:48.546349 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d468106-9fc5-4c0b-a97d-e1fdba74d13b service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:48.547617 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4d468106-9fc5-4c0b-a97d-e1fdba74d13b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:48.562508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d468106-9fc5-4c0b-a97d-e1fdba74d13b service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:48.569402 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847da118-d5d7-4bce-b6b7-dc1d843ef0bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:48.570303 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847da118-d5d7-4bce-b6b7-dc1d843ef0bb service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:48.570593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847da118-d5d7-4bce-b6b7-dc1d843ef0bb service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:48.570879 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847da118-d5d7-4bce-b6b7-dc1d843ef0bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:48.634121 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847da118-d5d7-4bce-b6b7-dc1d843ef0bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:48.635825 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847da118-d5d7-4bce-b6b7-dc1d843ef0bb service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:48.645966 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1220/2405] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:13:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:13:50.755017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a79e96-3371-44c0-861a-900b338e3b95 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:50.768872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a79e96-3371-44c0-861a-900b338e3b95 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:50.775827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3bf4b0-cd23-436c-b148-4f49b8683ffb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:50.776509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3bf4b0-cd23-436c-b148-4f49b8683ffb service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:50.776800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3bf4b0-cd23-436c-b148-4f49b8683ffb service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:50.777085 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3bf4b0-cd23-436c-b148-4f49b8683ffb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:50.778947 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4a3bf4b0-cd23-436c-b148-4f49b8683ffb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:50.793777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3bf4b0-cd23-436c-b148-4f49b8683ffb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:50.795490 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3bf4b0-cd23-436c-b148-4f49b8683ffb service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:50.808322 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1186/2406] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:13:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:13:56.851603 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847da118-d5d7-4bce-b6b7-dc1d843ef0bb service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:13:56.853866 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-847da118-d5d7-4bce-b6b7-dc1d843ef0bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:56.917109 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847da118-d5d7-4bce-b6b7-dc1d843ef0bb service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:13:56.921022 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1102d4-2750-4d13-9827-9d72273866a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:13:56.921668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1102d4-2750-4d13-9827-9d72273866a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:13:56.921955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1102d4-2750-4d13-9827-9d72273866a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:13:56.922232 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1102d4-2750-4d13-9827-9d72273866a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:13:56.924068 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ab1102d4-2750-4d13-9827-9d72273866a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:13:56.934775 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1102d4-2750-4d13-9827-9d72273866a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:13:56.936260 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1102d4-2750-4d13-9827-9d72273866a5 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:13:56.945572 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1221/2407] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:13:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:14:00.804172 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=8 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:14:06.945870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:14:20.158449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3bf4b0-cd23-436c-b148-4f49b8683ffb service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:20.170071 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3bf4b0-cd23-436c-b148-4f49b8683ffb service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:20.176655 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f1f9d-bc24-4f8d-926f-49cc19fb033d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:20.177293 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f1f9d-bc24-4f8d-926f-49cc19fb033d service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:20.177580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f1f9d-bc24-4f8d-926f-49cc19fb033d service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:20.177853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f1f9d-bc24-4f8d-926f-49cc19fb033d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:20.179695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e54f1f9d-bc24-4f8d-926f-49cc19fb033d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:20.242134 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f1f9d-bc24-4f8d-926f-49cc19fb033d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:20.244761 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f1f9d-bc24-4f8d-926f-49cc19fb033d service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:20.254505 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1187/2408] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:14:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:14:23.853090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0efc9f-da4b-4dbe-8493-0a5da442451a None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:23.853660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0efc9f-da4b-4dbe-8493-0a5da442451a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:23.853660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0efc9f-da4b-4dbe-8493-0a5da442451a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:23.872209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3a0efc9f-da4b-4dbe-8493-0a5da442451a None None] MFA Rules not processed for user `89bf838da11b41bc9b9f77c9f10a6bbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:14:23.873414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3a0efc9f-da4b-4dbe-8493-0a5da442451a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:23.880894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1222/2409] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:14:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:14:24.199710 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f1f9d-bc24-4f8d-926f-49cc19fb033d service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:24.209084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f1f9d-bc24-4f8d-926f-49cc19fb033d service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:24.212644 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f416b8-9c30-4221-b33c-fa5862282802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:24.213294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f416b8-9c30-4221-b33c-fa5862282802 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:24.213580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f416b8-9c30-4221-b33c-fa5862282802 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:24.213880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f416b8-9c30-4221-b33c-fa5862282802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:24.215693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a3f416b8-9c30-4221-b33c-fa5862282802 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:24.225382 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f416b8-9c30-4221-b33c-fa5862282802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:24.226903 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f416b8-9c30-4221-b33c-fa5862282802 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:24.236617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1188/2410] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:14:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:14:25.026258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0efc9f-da4b-4dbe-8493-0a5da442451a None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:25.027879 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3a0efc9f-da4b-4dbe-8493-0a5da442451a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:25.106055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0efc9f-da4b-4dbe-8493-0a5da442451a None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:25.111647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd87a6-87f3-4ca0-ad96-7f290995fc47 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c577fa8670e34449a80de0c8e0a15a8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:25.112255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd87a6-87f3-4ca0-ad96-7f290995fc47 service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:25.112544 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd87a6-87f3-4ca0-ad96-7f290995fc47 service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:25.112830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd87a6-87f3-4ca0-ad96-7f290995fc47 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:25.115254 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e4fd87a6-87f3-4ca0-ad96-7f290995fc47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:25.127646 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd87a6-87f3-4ca0-ad96-7f290995fc47 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:25.129930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd87a6-87f3-4ca0-ad96-7f290995fc47 service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:25.140485 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1223/2411] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:14:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:14:26.869254 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f625b9-7422-473a-b52d-4f2a5da87d2a None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:26.871447 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f625b9-7422-473a-b52d-4f2a5da87d2a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:26.871447 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f625b9-7422-473a-b52d-4f2a5da87d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:26.888284 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b2f625b9-7422-473a-b52d-4f2a5da87d2a None None] MFA Rules not processed for user `5aa7ff2fa5454bba9268e89c44b75de6`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:14:26.889473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b2f625b9-7422-473a-b52d-4f2a5da87d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:26.897367 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1189/2412] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:14:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:14:29.119742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd87a6-87f3-4ca0-ad96-7f290995fc47 service placement] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:29.121369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e4fd87a6-87f3-4ca0-ad96-7f290995fc47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:29.194696 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd87a6-87f3-4ca0-ad96-7f290995fc47 service placement] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:29.199995 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e8e10-f9a6-42e6-8979-2ba92b5b626e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:29.200879 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e8e10-f9a6-42e6-8979-2ba92b5b626e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:29.201163 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e8e10-f9a6-42e6-8979-2ba92b5b626e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:29.201440 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e8e10-f9a6-42e6-8979-2ba92b5b626e admin admin] PATH_INFO: `/v3/users/51c0e902dbb24a928b093f17a2cb1eb0` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:29.202635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e8e10-f9a6-42e6-8979-2ba92b5b626e admin admin] RBAC: Authorizing `identity:delete_user(user_id=51c0e902dbb24a928b093f17a2cb1eb0)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:29.204611 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e8e10-f9a6-42e6-8979-2ba92b5b626e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:29.377922 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d89e8e10-f9a6-42e6-8979-2ba92b5b626e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51c0e902dbb24a928b093f17a2cb1eb0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:14:29.378743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d89e8e10-f9a6-42e6-8979-2ba92b5b626e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51c0e902dbb24a928b093f17a2cb1eb0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:14:29.391445 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d89e8e10-f9a6-42e6-8979-2ba92b5b626e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51c0e902dbb24a928b093f17a2cb1eb0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:14:29.392149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d89e8e10-f9a6-42e6-8979-2ba92b5b626e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51c0e902dbb24a928b093f17a2cb1eb0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:14:29.400048 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d89e8e10-f9a6-42e6-8979-2ba92b5b626e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51c0e902dbb24a928b093f17a2cb1eb0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:14:29.513616 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1224/2413] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:14:29 2022] DELETE /identity/v3/users/51c0e902dbb24a928b093f17a2cb1eb0 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:14:29.760200 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f625b9-7422-473a-b52d-4f2a5da87d2a None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:29.774722 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f625b9-7422-473a-b52d-4f2a5da87d2a None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:29.841651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc251e-4a7a-4eed-91b9-67db3cefa9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:29.842523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc251e-4a7a-4eed-91b9-67db3cefa9c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:29.842809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc251e-4a7a-4eed-91b9-67db3cefa9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:29.843117 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc251e-4a7a-4eed-91b9-67db3cefa9c1 admin admin] PATH_INFO: `/v3/projects/d4d830928ef64831bf3b37562e4bcdf8` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:29.849421 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc251e-4a7a-4eed-91b9-67db3cefa9c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4d830928ef64831bf3b37562e4bcdf8)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:29.849421 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc251e-4a7a-4eed-91b9-67db3cefa9c1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:29.965055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d6dc251e-4a7a-4eed-91b9-67db3cefa9c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4d830928ef64831bf3b37562e4bcdf8', 'request_id': 'req-d6dc251e-4a7a-4eed-91b9-67db3cefa9c1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:14:30.049533 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d6dc251e-4a7a-4eed-91b9-67db3cefa9c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4d830928ef64831bf3b37562e4bcdf8', 'request_id': 'req-d6dc251e-4a7a-4eed-91b9-67db3cefa9c1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:14:30.056044 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d6dc251e-4a7a-4eed-91b9-67db3cefa9c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4d830928ef64831bf3b37562e4bcdf8', 'request_id': 'req-d6dc251e-4a7a-4eed-91b9-67db3cefa9c1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:14:30.071611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d6dc251e-4a7a-4eed-91b9-67db3cefa9c1 admin admin] The token cache is being invalidate because project d4d830928ef64831bf3b37562e4bcdf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:14:30.072045 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d6dc251e-4a7a-4eed-91b9-67db3cefa9c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:14:30.072988 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d6dc251e-4a7a-4eed-91b9-67db3cefa9c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:14:30.075145 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1190/2414] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:14:29 2022] DELETE /identity/v3/projects/d4d830928ef64831bf3b37562e4bcdf8 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:14:30.082798 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea7e35-01ed-49a6-bcb1-c73dd83a4d4a None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:30.083272 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea7e35-01ed-49a6-bcb1-c73dd83a4d4a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:30.083694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea7e35-01ed-49a6-bcb1-c73dd83a4d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:30.091574 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:14:30.105736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-afea7e35-01ed-49a6-bcb1-c73dd83a4d4a None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:14:30.107188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-afea7e35-01ed-49a6-bcb1-c73dd83a4d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:30.165688 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1225/2415] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:14:30.173250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe9cb2-fba8-48c8-a8a0-0ae731858dec None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:30.173542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe9cb2-fba8-48c8-a8a0-0ae731858dec None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:30.173820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe9cb2-fba8-48c8-a8a0-0ae731858dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:30.192334 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-cefe9cb2-fba8-48c8-a8a0-0ae731858dec None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:14:30.193538 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-cefe9cb2-fba8-48c8-a8a0-0ae731858dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:30.201328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1191/2416] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:14:30.205594 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea7e35-01ed-49a6-bcb1-c73dd83a4d4a None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:30.216401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea7e35-01ed-49a6-bcb1-c73dd83a4d4a None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:30.221034 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbf9b7-f5fb-439c-be98-8f5453e62b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:30.221691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbf9b7-f5fb-439c-be98-8f5453e62b50 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:30.221977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbf9b7-f5fb-439c-be98-8f5453e62b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:30.222253 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbf9b7-f5fb-439c-be98-8f5453e62b50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:30.223619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbf9b7-f5fb-439c-be98-8f5453e62b50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:14:30.223931 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbf9b7-f5fb-439c-be98-8f5453e62b50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:30.225557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbf9b7-f5fb-439c-be98-8f5453e62b50 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:30.237449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1226/2417] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:14:30.242371 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe9cb2-fba8-48c8-a8a0-0ae731858dec None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:30.246768 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:14:30.252092 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe9cb2-fba8-48c8-a8a0-0ae731858dec None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:30.265609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512c20c-a9cb-4535-94c0-c92d8a3a42f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:30.265609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512c20c-a9cb-4535-94c0-c92d8a3a42f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:30.265609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512c20c-a9cb-4535-94c0-c92d8a3a42f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:30.265609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512c20c-a9cb-4535-94c0-c92d8a3a42f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:30.266062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512c20c-a9cb-4535-94c0-c92d8a3a42f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:30.266062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512c20c-a9cb-4535-94c0-c92d8a3a42f1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:30.325925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1192/2418] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:14:30 2022] POST /identity/v3/projects => generated 381 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:14:30.330093 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbf9b7-f5fb-439c-be98-8f5453e62b50 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:30.340884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbf9b7-f5fb-439c-be98-8f5453e62b50 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:30.408213 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec1568-cf34-4c69-91cd-4c2c997c44d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:30.408892 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec1568-cf34-4c69-91cd-4c2c997c44d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:30.409180 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec1568-cf34-4c69-91cd-4c2c997c44d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:30.409458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec1568-cf34-4c69-91cd-4c2c997c44d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:30.410700 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec1568-cf34-4c69-91cd-4c2c997c44d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:30.412666 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec1568-cf34-4c69-91cd-4c2c997c44d4 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:30.516618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1227/2419] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:14:30 2022] POST /identity/v3/users => generated 291 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:14:30.518520 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512c20c-a9cb-4535-94c0-c92d8a3a42f1 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:30.531750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512c20c-a9cb-4535-94c0-c92d8a3a42f1 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:30.537136 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdfa53-f2d8-4d96-8eac-52cd3b2919b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:30.537813 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdfa53-f2d8-4d96-8eac-52cd3b2919b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:30.538145 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdfa53-f2d8-4d96-8eac-52cd3b2919b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:30.538424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdfa53-f2d8-4d96-8eac-52cd3b2919b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:30.539177 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdfa53-f2d8-4d96-8eac-52cd3b2919b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:30.540875 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdfa53-f2d8-4d96-8eac-52cd3b2919b1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:30.552318 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1193/2420] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:14:30.556300 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec1568-cf34-4c69-91cd-4c2c997c44d4 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:30.566330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec1568-cf34-4c69-91cd-4c2c997c44d4 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:30.575612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962baefb-e773-41b4-96b0-030d24f8798b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:30.575612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962baefb-e773-41b4-96b0-030d24f8798b admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:30.575612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962baefb-e773-41b4-96b0-030d24f8798b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:30.575612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962baefb-e773-41b4-96b0-030d24f8798b admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:30.576216 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962baefb-e773-41b4-96b0-030d24f8798b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:30.576216 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962baefb-e773-41b4-96b0-030d24f8798b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:30.591247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1228/2421] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:14:30.592475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdfa53-f2d8-4d96-8eac-52cd3b2919b1 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:30.602837 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdfa53-f2d8-4d96-8eac-52cd3b2919b1 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:30.607396 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c308d5-2663-48b6-88f7-9a885bf216f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:30.608280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c308d5-2663-48b6-88f7-9a885bf216f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:30.608562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c308d5-2663-48b6-88f7-9a885bf216f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:30.608830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c308d5-2663-48b6-88f7-9a885bf216f2 admin admin] PATH_INFO: `/v3/projects/c4b76c568f624909891661687bf457c4/users/7f8022891f234799b8636ec47f7078c2/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:30.624130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c308d5-2663-48b6-88f7-9a885bf216f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4b76c568f624909891661687bf457c4, user_id=7f8022891f234799b8636ec47f7078c2, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:30.626780 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c308d5-2663-48b6-88f7-9a885bf216f2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:30.684441 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1194/2422] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:14:30 2022] PUT /identity/v3/projects/c4b76c568f624909891661687bf457c4/users/7f8022891f234799b8636ec47f7078c2/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:14:30.689511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65c9fe-1119-461d-b4ba-184d011df041 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:30.694508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65c9fe-1119-461d-b4ba-184d011df041 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:30.694508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65c9fe-1119-461d-b4ba-184d011df041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:30.707770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ed65c9fe-1119-461d-b4ba-184d011df041 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:14:30.708972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ed65c9fe-1119-461d-b4ba-184d011df041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:30.772511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1229/2423] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:14:30.782235 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c308d5-2663-48b6-88f7-9a885bf216f2 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:30.783979 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-19c308d5-2663-48b6-88f7-9a885bf216f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:30.840947 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c308d5-2663-48b6-88f7-9a885bf216f2 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:30.844689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eaaba1-feba-4cae-b5cc-6198de3b5b29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:30.845338 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eaaba1-feba-4cae-b5cc-6198de3b5b29 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:30.845652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eaaba1-feba-4cae-b5cc-6198de3b5b29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:30.845926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eaaba1-feba-4cae-b5cc-6198de3b5b29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:30.855651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eaaba1-feba-4cae-b5cc-6198de3b5b29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:30.857159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eaaba1-feba-4cae-b5cc-6198de3b5b29 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:30.866793 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1195/2424] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:14:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:14:37.379654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357d5c7-9ca5-43e8-8372-1d304872a0fc None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:37.380206 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357d5c7-9ca5-43e8-8372-1d304872a0fc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:37.382416 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357d5c7-9ca5-43e8-8372-1d304872a0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:37.402515 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:14:37.411860 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e357d5c7-9ca5-43e8-8372-1d304872a0fc None None] MFA Rules not processed for user `7f8022891f234799b8636ec47f7078c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:14:37.413076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e357d5c7-9ca5-43e8-8372-1d304872a0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:37.478312 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1230/2425] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:14:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:14:37.482904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eaaba1-feba-4cae-b5cc-6198de3b5b29 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:37.492994 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eaaba1-feba-4cae-b5cc-6198de3b5b29 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:37.497747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018d738-9c9b-48a3-be35-9c2285e3ac83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:37.498410 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018d738-9c9b-48a3-be35-9c2285e3ac83 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:37.498700 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018d738-9c9b-48a3-be35-9c2285e3ac83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:37.498981 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018d738-9c9b-48a3-be35-9c2285e3ac83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:37.499766 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018d738-9c9b-48a3-be35-9c2285e3ac83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:37.502018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018d738-9c9b-48a3-be35-9c2285e3ac83 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:37.565523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1196/2426] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:14:37 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:14:37.569620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357d5c7-9ca5-43e8-8372-1d304872a0fc None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:37.582670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357d5c7-9ca5-43e8-8372-1d304872a0fc None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:37.639310 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e83e9-a475-4b54-b98b-d28cbe7c149f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:37.639995 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e83e9-a475-4b54-b98b-d28cbe7c149f admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:37.640290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e83e9-a475-4b54-b98b-d28cbe7c149f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:37.640567 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e83e9-a475-4b54-b98b-d28cbe7c149f admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:37.641253 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e83e9-a475-4b54-b98b-d28cbe7c149f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:37.643244 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e83e9-a475-4b54-b98b-d28cbe7c149f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:37.748124 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1231/2427] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:14:37 2022] POST /identity/v3/users => generated 299 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:14:37.752741 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018d738-9c9b-48a3-be35-9c2285e3ac83 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:37.763613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018d738-9c9b-48a3-be35-9c2285e3ac83 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:37.768177 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a8721-68be-41ae-834f-bf686786bcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:37.768839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a8721-68be-41ae-834f-bf686786bcea admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:37.769127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a8721-68be-41ae-834f-bf686786bcea admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:37.769406 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a8721-68be-41ae-834f-bf686786bcea admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:37.770132 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a8721-68be-41ae-834f-bf686786bcea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:37.771758 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a8721-68be-41ae-834f-bf686786bcea admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:37.782901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1197/2428] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:14:37.787483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e83e9-a475-4b54-b98b-d28cbe7c149f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:37.801138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e83e9-a475-4b54-b98b-d28cbe7c149f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:37.805907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292746bf-2edb-41da-9d4c-cd811f834109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:37.806802 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292746bf-2edb-41da-9d4c-cd811f834109 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:37.807085 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292746bf-2edb-41da-9d4c-cd811f834109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:37.807402 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292746bf-2edb-41da-9d4c-cd811f834109 admin admin] PATH_INFO: `/v3/projects/77ba62e66bda4f2aa7cc436186ffe794/users/c1fb89ab4d6d4433b9f8bbad00dc2e2f/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:37.825931 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292746bf-2edb-41da-9d4c-cd811f834109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ba62e66bda4f2aa7cc436186ffe794, user_id=c1fb89ab4d6d4433b9f8bbad00dc2e2f, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:37.826553 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292746bf-2edb-41da-9d4c-cd811f834109 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:37.919206 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1232/2429] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:14:37 2022] PUT /identity/v3/projects/77ba62e66bda4f2aa7cc436186ffe794/users/c1fb89ab4d6d4433b9f8bbad00dc2e2f/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:14:37.923623 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a8721-68be-41ae-834f-bf686786bcea admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:37.934595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a8721-68be-41ae-834f-bf686786bcea admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:37.993026 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca810c-84d5-478e-abdb-d8394c11ad89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:37.993666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca810c-84d5-478e-abdb-d8394c11ad89 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:37.994061 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca810c-84d5-478e-abdb-d8394c11ad89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:37.994450 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca810c-84d5-478e-abdb-d8394c11ad89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:37.995451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca810c-84d5-478e-abdb-d8394c11ad89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:37.997522 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca810c-84d5-478e-abdb-d8394c11ad89 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:38.014498 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1198/2430] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:14:38.016054 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292746bf-2edb-41da-9d4c-cd811f834109 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:38.028297 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292746bf-2edb-41da-9d4c-cd811f834109 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:38.033179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06912ac-06e2-4328-b5e8-0acb22b0df6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:38.034081 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06912ac-06e2-4328-b5e8-0acb22b0df6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:38.034372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06912ac-06e2-4328-b5e8-0acb22b0df6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:38.037571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06912ac-06e2-4328-b5e8-0acb22b0df6c admin admin] PATH_INFO: `/v3/projects/77ba62e66bda4f2aa7cc436186ffe794/users/c1fb89ab4d6d4433b9f8bbad00dc2e2f/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:38.037571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06912ac-06e2-4328-b5e8-0acb22b0df6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ba62e66bda4f2aa7cc436186ffe794, user_id=c1fb89ab4d6d4433b9f8bbad00dc2e2f, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:38.040143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06912ac-06e2-4328-b5e8-0acb22b0df6c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:38.051990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1233/2431] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:14:38 2022] PUT /identity/v3/projects/77ba62e66bda4f2aa7cc436186ffe794/users/c1fb89ab4d6d4433b9f8bbad00dc2e2f/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:14:43.252434 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca810c-84d5-478e-abdb-d8394c11ad89 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:43.255390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c9ca810c-84d5-478e-abdb-d8394c11ad89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:43.275625 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:14:43.342323 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca810c-84d5-478e-abdb-d8394c11ad89 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:43.416794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a8f02-c930-4246-af39-5da4085ecfaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:43.417571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a8f02-c930-4246-af39-5da4085ecfaf service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:43.417851 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a8f02-c930-4246-af39-5da4085ecfaf service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:43.418138 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a8f02-c930-4246-af39-5da4085ecfaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:43.420126 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e25a8f02-c930-4246-af39-5da4085ecfaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:43.436024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a8f02-c930-4246-af39-5da4085ecfaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:43.437836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a8f02-c930-4246-af39-5da4085ecfaf service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:43.451703 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1199/2432] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:14:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:14:44.890719 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67e967-e504-4b36-9e58-7a6dff631c92 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:44.891250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67e967-e504-4b36-9e58-7a6dff631c92 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:44.891364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67e967-e504-4b36-9e58-7a6dff631c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:44.921897 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-4c67e967-e504-4b36-9e58-7a6dff631c92 None None] MFA Rules not processed for user `c1fb89ab4d6d4433b9f8bbad00dc2e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:14:44.923294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4c67e967-e504-4b36-9e58-7a6dff631c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:44.973522 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:14:45.006788 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1234/2433] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:14:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:14:45.020068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a8f02-c930-4246-af39-5da4085ecfaf service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:45.021475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e25a8f02-c930-4246-af39-5da4085ecfaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:45.081853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a8f02-c930-4246-af39-5da4085ecfaf service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:45.085401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5c04f-1f75-40de-9b48-73f6020a51a4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5d76eae62b54521a95f36284118fada', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:45.086047 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5c04f-1f75-40de-9b48-73f6020a51a4 service glance] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:45.086326 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5c04f-1f75-40de-9b48-73f6020a51a4 service glance] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:45.086604 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5c04f-1f75-40de-9b48-73f6020a51a4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:45.133976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:14:45.145025 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d5c04f-1f75-40de-9b48-73f6020a51a4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:45.146565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d5c04f-1f75-40de-9b48-73f6020a51a4 service glance] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:45.156626 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1200/2434] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:14:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:14:45.495646 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85346d56-d193-4360-81dc-00ff32a80cce None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:45.495646 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85346d56-d193-4360-81dc-00ff32a80cce None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:45.495646 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85346d56-d193-4360-81dc-00ff32a80cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:45.516373 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-85346d56-d193-4360-81dc-00ff32a80cce None None] MFA Rules not processed for user `4ef0fad9a71d43bfbaba9c87dfed41c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:14:45.517609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-85346d56-d193-4360-81dc-00ff32a80cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:45.578328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1235/2435] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 14 19:14:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:14:45.762264 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5c04f-1f75-40de-9b48-73f6020a51a4 service glance] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:45.764200 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-15d5c04f-1f75-40de-9b48-73f6020a51a4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:45.844323 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5c04f-1f75-40de-9b48-73f6020a51a4 service glance] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:45.849927 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e187dc-ce4a-4c45-940d-4a9b71d983a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4368f68724f946eca53084a9ec7ff590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:45.850719 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e187dc-ce4a-4c45-940d-4a9b71d983a3 service swift] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:45.851016 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e187dc-ce4a-4c45-940d-4a9b71d983a3 service swift] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:45.851746 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e187dc-ce4a-4c45-940d-4a9b71d983a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:45.860934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e187dc-ce4a-4c45-940d-4a9b71d983a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:45.864200 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e187dc-ce4a-4c45-940d-4a9b71d983a3 service swift] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:45.872369 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1201/2436] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 14 19:14:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 19:14:46.516598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85346d56-d193-4360-81dc-00ff32a80cce None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:46.518881 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-85346d56-d193-4360-81dc-00ff32a80cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:46.606073 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85346d56-d193-4360-81dc-00ff32a80cce None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:46.616350 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6fb9c-1294-4332-ab52-92557fdc0413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:46.617357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6fb9c-1294-4332-ab52-92557fdc0413 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:46.618009 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6fb9c-1294-4332-ab52-92557fdc0413 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:46.619227 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6fb9c-1294-4332-ab52-92557fdc0413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:46.621279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-23e6fb9c-1294-4332-ab52-92557fdc0413 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:46.720729 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6fb9c-1294-4332-ab52-92557fdc0413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:46.725305 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6fb9c-1294-4332-ab52-92557fdc0413 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:46.740994 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1236/2437] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:14:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:14:49.660428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e187dc-ce4a-4c45-940d-4a9b71d983a3 service swift] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:49.672831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e187dc-ce4a-4c45-940d-4a9b71d983a3 service swift] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:49.679863 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336bf9f-e3e9-4a43-9d78-548a72d02d3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:49.680531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336bf9f-e3e9-4a43-9d78-548a72d02d3a service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:49.680823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336bf9f-e3e9-4a43-9d78-548a72d02d3a service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:49.681229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336bf9f-e3e9-4a43-9d78-548a72d02d3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:49.682922 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7336bf9f-e3e9-4a43-9d78-548a72d02d3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:49.696530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336bf9f-e3e9-4a43-9d78-548a72d02d3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:49.698221 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336bf9f-e3e9-4a43-9d78-548a72d02d3a service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:49.709873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1202/2438] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:14:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:14:49.945798 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6fb9c-1294-4332-ab52-92557fdc0413 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:14:49.949464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-23e6fb9c-1294-4332-ab52-92557fdc0413 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:49.974589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6fb9c-1294-4332-ab52-92557fdc0413 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:14:49.983261 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d71ef-5b1e-489e-bdb1-0b789dc16f5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:14:49.984160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d71ef-5b1e-489e-bdb1-0b789dc16f5b service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:49.984649 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d71ef-5b1e-489e-bdb1-0b789dc16f5b service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:49.985125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d71ef-5b1e-489e-bdb1-0b789dc16f5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:49.988416 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ba1d71ef-5b1e-489e-bdb1-0b789dc16f5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:50.041970 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:14:50.079980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d71ef-5b1e-489e-bdb1-0b789dc16f5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:14:50.081961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d71ef-5b1e-489e-bdb1-0b789dc16f5b service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:14:50.094340 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1237/2439] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:14:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:14:57.026388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efa184-9003-4354-a16f-7514be1e2cdc None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:14:57.026829 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efa184-9003-4354-a16f-7514be1e2cdc None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:14:57.027004 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efa184-9003-4354-a16f-7514be1e2cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:14:57.050600 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-38efa184-9003-4354-a16f-7514be1e2cdc None None] MFA Rules not processed for user `7f8022891f234799b8636ec47f7078c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:14:57.052882 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-38efa184-9003-4354-a16f-7514be1e2cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:14:57.066774 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1203/2440] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:14:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:15:00.088906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=14 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:15:00.634918 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e592ee-0d3f-4ca3-ab51-971bbb73e703 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:15:00.635144 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e592ee-0d3f-4ca3-ab51-971bbb73e703 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:15:00.635520 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e592ee-0d3f-4ca3-ab51-971bbb73e703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:15:00.653270 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-a7e592ee-0d3f-4ca3-ab51-971bbb73e703 None None] MFA Rules not processed for user `4ef0fad9a71d43bfbaba9c87dfed41c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:15:00.655905 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a7e592ee-0d3f-4ca3-ab51-971bbb73e703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:15:00.662598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1238/2441] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 14 19:15:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:15:00.676057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efa184-9003-4354-a16f-7514be1e2cdc None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:15:00.688499 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efa184-9003-4354-a16f-7514be1e2cdc None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:15:00.695209 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97c6e7-7b53-47f5-9b93-3757ccbcb6d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4368f68724f946eca53084a9ec7ff590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:15:00.696097 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97c6e7-7b53-47f5-9b93-3757ccbcb6d0 service swift] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:15:00.696445 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97c6e7-7b53-47f5-9b93-3757ccbcb6d0 service swift] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:15:00.696790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97c6e7-7b53-47f5-9b93-3757ccbcb6d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:15:00.705816 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97c6e7-7b53-47f5-9b93-3757ccbcb6d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:15:00.707614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97c6e7-7b53-47f5-9b93-3757ccbcb6d0 service swift] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:15:00.716514 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1204/2442] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 14 19:15:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 19:15:02.149831 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66291c03-3c98-4381-9554-114f532bea6e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:15:02.150945 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66291c03-3c98-4381-9554-114f532bea6e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:15:02.151268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66291c03-3c98-4381-9554-114f532bea6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:15:02.170949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-66291c03-3c98-4381-9554-114f532bea6e None None] MFA Rules not processed for user `b72067d66a7649e7b8c120dd4d3fe635`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:15:02.172238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-66291c03-3c98-4381-9554-114f532bea6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:15:02.184217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1239/2443] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:15:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:15:10.713017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=52 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:15:12.164896 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=6 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:15:39.445927 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97c6e7-7b53-47f5-9b93-3757ccbcb6d0 service swift] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:15:39.455835 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97c6e7-7b53-47f5-9b93-3757ccbcb6d0 service swift] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:15:39.459410 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c7139-2edd-4c6f-a102-2843b89baf26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:15:39.460050 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c7139-2edd-4c6f-a102-2843b89baf26 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:15:39.460337 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c7139-2edd-4c6f-a102-2843b89baf26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:15:39.460618 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c7139-2edd-4c6f-a102-2843b89baf26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:15:39.469633 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c7139-2edd-4c6f-a102-2843b89baf26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:15:39.471100 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c7139-2edd-4c6f-a102-2843b89baf26 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:15:39.481469 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1205/2444] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:15:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:15:49.477143 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=6 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:01.805126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66291c03-3c98-4381-9554-114f532bea6e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:01.828105 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66291c03-3c98-4381-9554-114f532bea6e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:01.833319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562889a-2d8b-4605-9746-a1794189b8be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:01.833967 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562889a-2d8b-4605-9746-a1794189b8be service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:01.834258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562889a-2d8b-4605-9746-a1794189b8be service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:01.834540 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562889a-2d8b-4605-9746-a1794189b8be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:01.836437 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8562889a-2d8b-4605-9746-a1794189b8be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:01.962763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562889a-2d8b-4605-9746-a1794189b8be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:01.964175 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562889a-2d8b-4605-9746-a1794189b8be service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:01.984907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1240/2445] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:16:04.949900 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c7139-2edd-4c6f-a102-2843b89baf26 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:04.952193 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-140c7139-2edd-4c6f-a102-2843b89baf26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:05.137114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c7139-2edd-4c6f-a102-2843b89baf26 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:05.149388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7492f1c-ee23-4601-b610-f0e5c81cab02 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c577fa8670e34449a80de0c8e0a15a8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:05.151469 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7492f1c-ee23-4601-b610-f0e5c81cab02 service placement] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:05.151765 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7492f1c-ee23-4601-b610-f0e5c81cab02 service placement] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:05.153388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7492f1c-ee23-4601-b610-f0e5c81cab02 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:05.155118 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a7492f1c-ee23-4601-b610-f0e5c81cab02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:05.176073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7492f1c-ee23-4601-b610-f0e5c81cab02 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:05.178471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7492f1c-ee23-4601-b610-f0e5c81cab02 service placement] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:05.191791 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1206/2446] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:16:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:16:06.715816 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562889a-2d8b-4605-9746-a1794189b8be service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:06.717656 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8562889a-2d8b-4605-9746-a1794189b8be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:06.734147 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562889a-2d8b-4605-9746-a1794189b8be service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:06.740203 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9a5e3-3b6a-456f-9aca-a3cf6e346d01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c577fa8670e34449a80de0c8e0a15a8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:06.741250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9a5e3-3b6a-456f-9aca-a3cf6e346d01 service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:06.742347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9a5e3-3b6a-456f-9aca-a3cf6e346d01 service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:06.742740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9a5e3-3b6a-456f-9aca-a3cf6e346d01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:06.744769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4aa9a5e3-3b6a-456f-9aca-a3cf6e346d01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:06.765568 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9a5e3-3b6a-456f-9aca-a3cf6e346d01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:06.767273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9a5e3-3b6a-456f-9aca-a3cf6e346d01 service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:06.778081 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1241/2447] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:16:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:16:06.827897 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7492f1c-ee23-4601-b610-f0e5c81cab02 service placement] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:06.829333 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a7492f1c-ee23-4601-b610-f0e5c81cab02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:06.850437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7492f1c-ee23-4601-b610-f0e5c81cab02 service placement] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:06.856068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7120a7-4c73-466d-bc66-c3558dead7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:06.856909 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7120a7-4c73-466d-bc66-c3558dead7f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:06.857178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7120a7-4c73-466d-bc66-c3558dead7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:06.857459 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7120a7-4c73-466d-bc66-c3558dead7f5 admin admin] PATH_INFO: `/v3/users/5aa7ff2fa5454bba9268e89c44b75de6` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:06.858711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7120a7-4c73-466d-bc66-c3558dead7f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aa7ff2fa5454bba9268e89c44b75de6)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:06.861287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7120a7-4c73-466d-bc66-c3558dead7f5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:07.051541 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3a7120a7-4c73-466d-bc66-c3558dead7f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aa7ff2fa5454bba9268e89c44b75de6'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:07.058632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3a7120a7-4c73-466d-bc66-c3558dead7f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aa7ff2fa5454bba9268e89c44b75de6'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:07.059337 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3a7120a7-4c73-466d-bc66-c3558dead7f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aa7ff2fa5454bba9268e89c44b75de6'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:07.059947 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3a7120a7-4c73-466d-bc66-c3558dead7f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aa7ff2fa5454bba9268e89c44b75de6'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:07.071696 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3a7120a7-4c73-466d-bc66-c3558dead7f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aa7ff2fa5454bba9268e89c44b75de6'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:07.128907 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1207/2448] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:16:06 2022] DELETE /identity/v3/users/5aa7ff2fa5454bba9268e89c44b75de6 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:07.428403 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9a5e3-3b6a-456f-9aca-a3cf6e346d01 service placement] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:07.439479 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9a5e3-3b6a-456f-9aca-a3cf6e346d01 service placement] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:07.507416 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f701db0-3137-40cc-b7ab-f253f159e2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:07.508391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f701db0-3137-40cc-b7ab-f253f159e2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:07.508671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f701db0-3137-40cc-b7ab-f253f159e2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:07.508942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f701db0-3137-40cc-b7ab-f253f159e2fb admin admin] PATH_INFO: `/v3/projects/01bc27a5fcbb4d9d94cab5f5906c43fa` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:07.510211 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f701db0-3137-40cc-b7ab-f253f159e2fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=01bc27a5fcbb4d9d94cab5f5906c43fa)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:07.512110 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f701db0-3137-40cc-b7ab-f253f159e2fb admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:07.529751 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:07.619829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3f701db0-3137-40cc-b7ab-f253f159e2fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01bc27a5fcbb4d9d94cab5f5906c43fa', 'request_id': 'req-3f701db0-3137-40cc-b7ab-f253f159e2fb'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:07.683521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3f701db0-3137-40cc-b7ab-f253f159e2fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01bc27a5fcbb4d9d94cab5f5906c43fa', 'request_id': 'req-3f701db0-3137-40cc-b7ab-f253f159e2fb'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:07.690549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3f701db0-3137-40cc-b7ab-f253f159e2fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01bc27a5fcbb4d9d94cab5f5906c43fa', 'request_id': 'req-3f701db0-3137-40cc-b7ab-f253f159e2fb'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:07.702094 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3f701db0-3137-40cc-b7ab-f253f159e2fb admin admin] The token cache is being invalidate because project 01bc27a5fcbb4d9d94cab5f5906c43fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:16:07.702409 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3f701db0-3137-40cc-b7ab-f253f159e2fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:07.703032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3f701db0-3137-40cc-b7ab-f253f159e2fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:07.705256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1242/2449] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:16:07 2022] DELETE /identity/v3/projects/01bc27a5fcbb4d9d94cab5f5906c43fa => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:07.710228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf99bc-f90e-4825-ab30-0103cf9ab21b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:07.710523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf99bc-f90e-4825-ab30-0103cf9ab21b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:07.711756 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf99bc-f90e-4825-ab30-0103cf9ab21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:07.714352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:07.772507 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-8cbf99bc-f90e-4825-ab30-0103cf9ab21b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:07.773644 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8cbf99bc-f90e-4825-ab30-0103cf9ab21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:07.840354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1208/2450] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:16:07.842251 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e4ab0-5ccd-478a-8493-cd95c2e117d8 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:07.842696 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e4ab0-5ccd-478a-8493-cd95c2e117d8 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:07.842899 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e4ab0-5ccd-478a-8493-cd95c2e117d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:07.862546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-fe3e4ab0-5ccd-478a-8493-cd95c2e117d8 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:07.863808 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-fe3e4ab0-5ccd-478a-8493-cd95c2e117d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:07.872030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1243/2451] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:16:07.877018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf99bc-f90e-4825-ab30-0103cf9ab21b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:07.887352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf99bc-f90e-4825-ab30-0103cf9ab21b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:07.893473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf23f4e-546c-4c09-9ec4-0a297448ac57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:07.894152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf23f4e-546c-4c09-9ec4-0a297448ac57 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:07.894443 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf23f4e-546c-4c09-9ec4-0a297448ac57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:07.894726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf23f4e-546c-4c09-9ec4-0a297448ac57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:07.895502 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf23f4e-546c-4c09-9ec4-0a297448ac57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:16:07.895818 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf23f4e-546c-4c09-9ec4-0a297448ac57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:07.897432 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf23f4e-546c-4c09-9ec4-0a297448ac57 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:07.910210 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1209/2452] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:16:07.914628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e4ab0-5ccd-478a-8493-cd95c2e117d8 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:07.924207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e4ab0-5ccd-478a-8493-cd95c2e117d8 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:07.928902 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47eb23-5416-4f18-927d-71afe3e39961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:07.929551 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47eb23-5416-4f18-927d-71afe3e39961 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:07.929834 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47eb23-5416-4f18-927d-71afe3e39961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:07.930108 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47eb23-5416-4f18-927d-71afe3e39961 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:07.930826 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47eb23-5416-4f18-927d-71afe3e39961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:07.932773 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47eb23-5416-4f18-927d-71afe3e39961 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:08.033806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1244/2453] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:16:07 2022] POST /identity/v3/projects => generated 405 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:16:08.037648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf23f4e-546c-4c09-9ec4-0a297448ac57 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:08.048186 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf23f4e-546c-4c09-9ec4-0a297448ac57 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:08.109692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80316e-02cd-499a-8a91-b26c34a28bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:08.110890 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80316e-02cd-499a-8a91-b26c34a28bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:08.111190 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80316e-02cd-499a-8a91-b26c34a28bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:08.111485 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80316e-02cd-499a-8a91-b26c34a28bf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:08.112869 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80316e-02cd-499a-8a91-b26c34a28bf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:08.114124 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80316e-02cd-499a-8a91-b26c34a28bf8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:08.208574 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1210/2454] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:16:08 2022] POST /identity/v3/users => generated 303 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:16:08.213204 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47eb23-5416-4f18-927d-71afe3e39961 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:08.224469 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47eb23-5416-4f18-927d-71afe3e39961 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:08.229332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcce902-4224-4fa1-bec6-c64645bd39d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:08.230090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcce902-4224-4fa1-bec6-c64645bd39d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:08.230448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcce902-4224-4fa1-bec6-c64645bd39d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:08.230797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcce902-4224-4fa1-bec6-c64645bd39d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:08.231640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcce902-4224-4fa1-bec6-c64645bd39d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:08.233368 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcce902-4224-4fa1-bec6-c64645bd39d7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:08.245494 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1245/2455] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:16:08.249593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80316e-02cd-499a-8a91-b26c34a28bf8 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:08.260701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80316e-02cd-499a-8a91-b26c34a28bf8 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:08.265486 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37929cbc-32a0-41a1-968e-e34512eb325d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:08.266156 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37929cbc-32a0-41a1-968e-e34512eb325d admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:08.266447 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37929cbc-32a0-41a1-968e-e34512eb325d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:08.266726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37929cbc-32a0-41a1-968e-e34512eb325d admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:08.267491 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37929cbc-32a0-41a1-968e-e34512eb325d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:08.269131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37929cbc-32a0-41a1-968e-e34512eb325d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:08.280131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1211/2456] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:16:08.284381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcce902-4224-4fa1-bec6-c64645bd39d7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:08.294812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcce902-4224-4fa1-bec6-c64645bd39d7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:08.300073 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3c2e7-5976-4d11-b9ec-4c9cf95f14a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:08.302022 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3c2e7-5976-4d11-b9ec-4c9cf95f14a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:08.302022 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3c2e7-5976-4d11-b9ec-4c9cf95f14a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:08.302022 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3c2e7-5976-4d11-b9ec-4c9cf95f14a0 admin admin] PATH_INFO: `/v3/projects/502eaf92ca764c578a43e48949e33dc1/users/70269c88f6604f20bf4d8848f5c0ace1/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:08.319346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3c2e7-5976-4d11-b9ec-4c9cf95f14a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=502eaf92ca764c578a43e48949e33dc1, user_id=70269c88f6604f20bf4d8848f5c0ace1, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:08.326501 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3c2e7-5976-4d11-b9ec-4c9cf95f14a0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:08.415334 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1246/2457] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:16:08 2022] PUT /identity/v3/projects/502eaf92ca764c578a43e48949e33dc1/users/70269c88f6604f20bf4d8848f5c0ace1/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:08.421263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232d7d3-a516-4718-b9a0-94fb0ec3943c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:08.421546 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232d7d3-a516-4718-b9a0-94fb0ec3943c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:08.421831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232d7d3-a516-4718-b9a0-94fb0ec3943c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:08.440812 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-8232d7d3-a516-4718-b9a0-94fb0ec3943c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:08.442019 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8232d7d3-a516-4718-b9a0-94fb0ec3943c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:08.473825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:08.502167 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1212/2458] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:16:08.511844 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3c2e7-5976-4d11-b9ec-4c9cf95f14a0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:08.513536 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2ef3c2e7-5976-4d11-b9ec-4c9cf95f14a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:08.595146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3c2e7-5976-4d11-b9ec-4c9cf95f14a0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:08.600013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999fc24-a71d-4b78-9ed7-f405dbc2b0dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:08.602604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999fc24-a71d-4b78-9ed7-f405dbc2b0dc service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:08.602604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999fc24-a71d-4b78-9ed7-f405dbc2b0dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:08.602604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999fc24-a71d-4b78-9ed7-f405dbc2b0dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:08.611595 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999fc24-a71d-4b78-9ed7-f405dbc2b0dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:08.613100 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999fc24-a71d-4b78-9ed7-f405dbc2b0dc service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:08.624676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1247/2459] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:16:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:16:09.484160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f708b-353d-4544-bb7a-81a40d4aa201 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:09.484660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f708b-353d-4544-bb7a-81a40d4aa201 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:09.484760 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f708b-353d-4544-bb7a-81a40d4aa201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:09.502979 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-8f3f708b-353d-4544-bb7a-81a40d4aa201 None None] MFA Rules not processed for user `7d5fa180ee554e8b8e46321892969489`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:09.504212 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8f3f708b-353d-4544-bb7a-81a40d4aa201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:09.562283 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1213/2460] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:16:09.854599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958cc53-3e89-4660-9d86-02b9cf26d0d3 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:09.855146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958cc53-3e89-4660-9d86-02b9cf26d0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:09.855421 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958cc53-3e89-4660-9d86-02b9cf26d0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:09.888440 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-1958cc53-3e89-4660-9d86-02b9cf26d0d3 None None] MFA Rules not processed for user `70269c88f6604f20bf4d8848f5c0ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:09.889671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1958cc53-3e89-4660-9d86-02b9cf26d0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:09.902048 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:09.964692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1248/2461] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:09 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:16:09.969982 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f708b-353d-4544-bb7a-81a40d4aa201 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:09.980663 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f708b-353d-4544-bb7a-81a40d4aa201 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:09.985727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55612a63-2746-43cf-b371-8f55d7c340d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:09.986381 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55612a63-2746-43cf-b371-8f55d7c340d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:09.986666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55612a63-2746-43cf-b371-8f55d7c340d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:09.986941 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55612a63-2746-43cf-b371-8f55d7c340d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:09.987663 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55612a63-2746-43cf-b371-8f55d7c340d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:09.989597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55612a63-2746-43cf-b371-8f55d7c340d3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:10.079222 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1214/2462] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:16:09 2022] POST /identity/v3/projects => generated 405 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:16:10.083232 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958cc53-3e89-4660-9d86-02b9cf26d0d3 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:10.095094 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958cc53-3e89-4660-9d86-02b9cf26d0d3 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:10.154195 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae559ccf-2a0f-4b5f-8ec1-344d0c055f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:10.154838 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae559ccf-2a0f-4b5f-8ec1-344d0c055f6b admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:10.155128 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae559ccf-2a0f-4b5f-8ec1-344d0c055f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:10.155448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae559ccf-2a0f-4b5f-8ec1-344d0c055f6b admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:10.156145 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae559ccf-2a0f-4b5f-8ec1-344d0c055f6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:10.158602 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae559ccf-2a0f-4b5f-8ec1-344d0c055f6b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:10.231713 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1249/2463] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:16:10 2022] POST /identity/v3/users => generated 309 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:16:10.236273 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55612a63-2746-43cf-b371-8f55d7c340d3 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:10.247038 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55612a63-2746-43cf-b371-8f55d7c340d3 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:10.251829 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1499f9a-00c6-4e7e-947c-b29b014bb04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:10.252513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1499f9a-00c6-4e7e-947c-b29b014bb04b admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:10.252782 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1499f9a-00c6-4e7e-947c-b29b014bb04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:10.253085 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1499f9a-00c6-4e7e-947c-b29b014bb04b admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:10.253808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1499f9a-00c6-4e7e-947c-b29b014bb04b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:10.255502 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1499f9a-00c6-4e7e-947c-b29b014bb04b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:10.267332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1215/2464] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:16:10.271285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae559ccf-2a0f-4b5f-8ec1-344d0c055f6b admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:10.283603 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae559ccf-2a0f-4b5f-8ec1-344d0c055f6b admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:10.288692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1594e4-d9ad-485c-9466-5dac7a3d1b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:10.289589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1594e4-d9ad-485c-9466-5dac7a3d1b35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:10.289879 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1594e4-d9ad-485c-9466-5dac7a3d1b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:10.290161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1594e4-d9ad-485c-9466-5dac7a3d1b35 admin admin] PATH_INFO: `/v3/projects/baa6607444874b10b6f4d53b4e35f366/users/8ab85b14d25542a4a48ff488bb76ac59/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:10.305868 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1594e4-d9ad-485c-9466-5dac7a3d1b35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baa6607444874b10b6f4d53b4e35f366, user_id=8ab85b14d25542a4a48ff488bb76ac59, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:10.308599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1594e4-d9ad-485c-9466-5dac7a3d1b35 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:10.370445 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1250/2465] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:16:10 2022] PUT /identity/v3/projects/baa6607444874b10b6f4d53b4e35f366/users/8ab85b14d25542a4a48ff488bb76ac59/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:10.374388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1499f9a-00c6-4e7e-947c-b29b014bb04b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:10.385176 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1499f9a-00c6-4e7e-947c-b29b014bb04b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:10.449413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683cfa89-eead-4aec-b192-b20c8f00d892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:10.450088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683cfa89-eead-4aec-b192-b20c8f00d892 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:10.450374 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683cfa89-eead-4aec-b192-b20c8f00d892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:10.450655 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683cfa89-eead-4aec-b192-b20c8f00d892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:10.451444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683cfa89-eead-4aec-b192-b20c8f00d892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:10.453038 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683cfa89-eead-4aec-b192-b20c8f00d892 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:10.463549 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1216/2466] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:16:10.467779 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1594e4-d9ad-485c-9466-5dac7a3d1b35 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:10.478913 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1594e4-d9ad-485c-9466-5dac7a3d1b35 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:10.483578 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f6e45-bd08-4fec-9051-ca5a03e26b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:10.484561 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f6e45-bd08-4fec-9051-ca5a03e26b56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:10.484898 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f6e45-bd08-4fec-9051-ca5a03e26b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:10.485323 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f6e45-bd08-4fec-9051-ca5a03e26b56 admin admin] PATH_INFO: `/v3/projects/baa6607444874b10b6f4d53b4e35f366/users/8ab85b14d25542a4a48ff488bb76ac59/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:10.487405 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f6e45-bd08-4fec-9051-ca5a03e26b56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baa6607444874b10b6f4d53b4e35f366, user_id=8ab85b14d25542a4a48ff488bb76ac59, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:10.490155 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f6e45-bd08-4fec-9051-ca5a03e26b56 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:10.501334 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1251/2467] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:16:10 2022] PUT /identity/v3/projects/baa6607444874b10b6f4d53b4e35f366/users/8ab85b14d25542a4a48ff488bb76ac59/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:11.771189 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8db4f-163c-4443-a28d-d13260dbddd2 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:11.771739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8db4f-163c-4443-a28d-d13260dbddd2 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:11.771841 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8db4f-163c-4443-a28d-d13260dbddd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:11.799914 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d0d8db4f-163c-4443-a28d-d13260dbddd2 None None] MFA Rules not processed for user `8ab85b14d25542a4a48ff488bb76ac59`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:11.801127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d0d8db4f-163c-4443-a28d-d13260dbddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:11.836779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:11.872857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1217/2468] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:11 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:16:13.386671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f6e45-bd08-4fec-9051-ca5a03e26b56 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:13.388740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-612f6e45-bd08-4fec-9051-ca5a03e26b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:13.464589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f6e45-bd08-4fec-9051-ca5a03e26b56 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:13.470461 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e8085-082d-4e20-b43c-8a29c42f7a2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c577fa8670e34449a80de0c8e0a15a8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:13.471108 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e8085-082d-4e20-b43c-8a29c42f7a2d service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:13.471421 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e8085-082d-4e20-b43c-8a29c42f7a2d service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:13.471702 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e8085-082d-4e20-b43c-8a29c42f7a2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:13.473449 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-042e8085-082d-4e20-b43c-8a29c42f7a2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:13.481459 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:13.485258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e8085-082d-4e20-b43c-8a29c42f7a2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:13.486869 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e8085-082d-4e20-b43c-8a29c42f7a2d service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:13.496751 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1252/2469] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:16:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:16:13.552496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8db4f-163c-4443-a28d-d13260dbddd2 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:13.624757 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8db4f-163c-4443-a28d-d13260dbddd2 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:13.631492 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7e5af-ebb5-4e97-aa0a-5febe2d295e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:13.633002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7e5af-ebb5-4e97-aa0a-5febe2d295e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:13.633300 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7e5af-ebb5-4e97-aa0a-5febe2d295e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:13.634323 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7e5af-ebb5-4e97-aa0a-5febe2d295e6 admin admin] PATH_INFO: `/v3/users/70269c88f6604f20bf4d8848f5c0ace1` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:13.636193 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7e5af-ebb5-4e97-aa0a-5febe2d295e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70269c88f6604f20bf4d8848f5c0ace1)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:13.638299 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7e5af-ebb5-4e97-aa0a-5febe2d295e6 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:13.810559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b3b7e5af-ebb5-4e97-aa0a-5febe2d295e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70269c88f6604f20bf4d8848f5c0ace1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:13.830450 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b3b7e5af-ebb5-4e97-aa0a-5febe2d295e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70269c88f6604f20bf4d8848f5c0ace1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:13.831393 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b3b7e5af-ebb5-4e97-aa0a-5febe2d295e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70269c88f6604f20bf4d8848f5c0ace1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:13.832269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b3b7e5af-ebb5-4e97-aa0a-5febe2d295e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70269c88f6604f20bf4d8848f5c0ace1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:13.849019 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b3b7e5af-ebb5-4e97-aa0a-5febe2d295e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70269c88f6604f20bf4d8848f5c0ace1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:13.922430 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1218/2470] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:16:13 2022] DELETE /identity/v3/users/70269c88f6604f20bf4d8848f5c0ace1 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:14.418711 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e8085-082d-4e20-b43c-8a29c42f7a2d service placement] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:14.442126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e8085-082d-4e20-b43c-8a29c42f7a2d service placement] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:14.501466 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36b31c-61fe-4cef-a67c-e714f8e6a5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:14.502355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36b31c-61fe-4cef-a67c-e714f8e6a5ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:14.502731 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36b31c-61fe-4cef-a67c-e714f8e6a5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:14.503016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36b31c-61fe-4cef-a67c-e714f8e6a5ad admin admin] PATH_INFO: `/v3/projects/502eaf92ca764c578a43e48949e33dc1` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:14.504260 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36b31c-61fe-4cef-a67c-e714f8e6a5ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=502eaf92ca764c578a43e48949e33dc1)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:14.506368 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36b31c-61fe-4cef-a67c-e714f8e6a5ad admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:14.617192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6e36b31c-61fe-4cef-a67c-e714f8e6a5ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '502eaf92ca764c578a43e48949e33dc1', 'request_id': 'req-6e36b31c-61fe-4cef-a67c-e714f8e6a5ad'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:14.665640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6e36b31c-61fe-4cef-a67c-e714f8e6a5ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '502eaf92ca764c578a43e48949e33dc1', 'request_id': 'req-6e36b31c-61fe-4cef-a67c-e714f8e6a5ad'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:14.674226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6e36b31c-61fe-4cef-a67c-e714f8e6a5ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '502eaf92ca764c578a43e48949e33dc1', 'request_id': 'req-6e36b31c-61fe-4cef-a67c-e714f8e6a5ad'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:14.687028 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6e36b31c-61fe-4cef-a67c-e714f8e6a5ad admin admin] The token cache is being invalidate because project 502eaf92ca764c578a43e48949e33dc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:16:14.687492 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6e36b31c-61fe-4cef-a67c-e714f8e6a5ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:14.688254 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6e36b31c-61fe-4cef-a67c-e714f8e6a5ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:14.689988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1253/2471] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:16:14 2022] DELETE /identity/v3/projects/502eaf92ca764c578a43e48949e33dc1 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:14.695143 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7e5af-ebb5-4e97-aa0a-5febe2d295e6 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:14.697250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b3b7e5af-ebb5-4e97-aa0a-5febe2d295e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:14.711911 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7e5af-ebb5-4e97-aa0a-5febe2d295e6 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:14.765927 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:14.777233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f243c0e-4901-45a0-8026-cffd71615fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:14.778408 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f243c0e-4901-45a0-8026-cffd71615fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:14.778797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f243c0e-4901-45a0-8026-cffd71615fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:14.779084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f243c0e-4901-45a0-8026-cffd71615fe1 admin admin] PATH_INFO: `/v3/users/8ab85b14d25542a4a48ff488bb76ac59` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:14.780423 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f243c0e-4901-45a0-8026-cffd71615fe1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ab85b14d25542a4a48ff488bb76ac59)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:14.782943 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f243c0e-4901-45a0-8026-cffd71615fe1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:14.960914 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2f243c0e-4901-45a0-8026-cffd71615fe1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ab85b14d25542a4a48ff488bb76ac59'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:14.969156 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2f243c0e-4901-45a0-8026-cffd71615fe1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ab85b14d25542a4a48ff488bb76ac59'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:14.970726 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2f243c0e-4901-45a0-8026-cffd71615fe1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ab85b14d25542a4a48ff488bb76ac59'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:14.971387 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2f243c0e-4901-45a0-8026-cffd71615fe1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ab85b14d25542a4a48ff488bb76ac59'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:14.985512 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2f243c0e-4901-45a0-8026-cffd71615fe1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ab85b14d25542a4a48ff488bb76ac59'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:15.123035 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1219/2472] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:16:14 2022] DELETE /identity/v3/users/8ab85b14d25542a4a48ff488bb76ac59 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:15.376595 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36b31c-61fe-4cef-a67c-e714f8e6a5ad admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:15.378141 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6e36b31c-61fe-4cef-a67c-e714f8e6a5ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:15.392678 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36b31c-61fe-4cef-a67c-e714f8e6a5ad admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:15.452790 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4d59e-f3b0-4492-9cda-11fbc4ada8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:15.454315 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4d59e-f3b0-4492-9cda-11fbc4ada8ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:15.454710 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4d59e-f3b0-4492-9cda-11fbc4ada8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:15.454993 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4d59e-f3b0-4492-9cda-11fbc4ada8ff admin admin] PATH_INFO: `/v3/projects/baa6607444874b10b6f4d53b4e35f366` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:15.456243 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4d59e-f3b0-4492-9cda-11fbc4ada8ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=baa6607444874b10b6f4d53b4e35f366)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:15.458188 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4d59e-f3b0-4492-9cda-11fbc4ada8ff admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:15.536285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:15.553244 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3de4d59e-f3b0-4492-9cda-11fbc4ada8ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baa6607444874b10b6f4d53b4e35f366', 'request_id': 'req-3de4d59e-f3b0-4492-9cda-11fbc4ada8ff'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:15.607958 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3de4d59e-f3b0-4492-9cda-11fbc4ada8ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baa6607444874b10b6f4d53b4e35f366', 'request_id': 'req-3de4d59e-f3b0-4492-9cda-11fbc4ada8ff'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:15.615086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3de4d59e-f3b0-4492-9cda-11fbc4ada8ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baa6607444874b10b6f4d53b4e35f366', 'request_id': 'req-3de4d59e-f3b0-4492-9cda-11fbc4ada8ff'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:15.626810 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3de4d59e-f3b0-4492-9cda-11fbc4ada8ff admin admin] The token cache is being invalidate because project baa6607444874b10b6f4d53b4e35f366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:16:15.627095 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3de4d59e-f3b0-4492-9cda-11fbc4ada8ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:15.627765 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3de4d59e-f3b0-4492-9cda-11fbc4ada8ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:15.629419 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1254/2473] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:16:15 2022] DELETE /identity/v3/projects/baa6607444874b10b6f4d53b4e35f366 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:15.634334 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e80c7-0f38-455b-8a44-00d41af60a40 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:15.634628 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e80c7-0f38-455b-8a44-00d41af60a40 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:15.634908 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e80c7-0f38-455b-8a44-00d41af60a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:15.653302 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d42e80c7-0f38-455b-8a44-00d41af60a40 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:15.654503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d42e80c7-0f38-455b-8a44-00d41af60a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:15.713104 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1220/2474] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:16:15.722798 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe1b99-651a-48aa-a5ec-b4f69263662c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:15.723094 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe1b99-651a-48aa-a5ec-b4f69263662c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:15.723421 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe1b99-651a-48aa-a5ec-b4f69263662c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:15.741312 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-bcfe1b99-651a-48aa-a5ec-b4f69263662c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:15.742525 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bcfe1b99-651a-48aa-a5ec-b4f69263662c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:15.750406 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1255/2475] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:16:15.754558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e80c7-0f38-455b-8a44-00d41af60a40 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:15.764371 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e80c7-0f38-455b-8a44-00d41af60a40 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:15.769366 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c6b43-459e-4a78-a336-83060f74161c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:15.770041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c6b43-459e-4a78-a336-83060f74161c admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:15.770323 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c6b43-459e-4a78-a336-83060f74161c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:15.770599 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c6b43-459e-4a78-a336-83060f74161c admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:15.771372 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c6b43-459e-4a78-a336-83060f74161c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:16:15.771681 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c6b43-459e-4a78-a336-83060f74161c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:15.773273 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c6b43-459e-4a78-a336-83060f74161c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:15.785440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1221/2476] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:16:15.789491 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe1b99-651a-48aa-a5ec-b4f69263662c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:15.798559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe1b99-651a-48aa-a5ec-b4f69263662c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:15.803136 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e95d5-9d27-4a91-adc2-8f7443761a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:15.803812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e95d5-9d27-4a91-adc2-8f7443761a9f admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:15.804097 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e95d5-9d27-4a91-adc2-8f7443761a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:15.804372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e95d5-9d27-4a91-adc2-8f7443761a9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:15.805061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e95d5-9d27-4a91-adc2-8f7443761a9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:15.806940 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e95d5-9d27-4a91-adc2-8f7443761a9f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:15.878635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1256/2477] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:16:15 2022] POST /identity/v3/projects => generated 387 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:16:15.882841 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c6b43-459e-4a78-a336-83060f74161c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:15.893578 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c6b43-459e-4a78-a336-83060f74161c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:15.945068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:15.950918 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e792f-813a-440a-82a2-4675fe26085f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:15.951627 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e792f-813a-440a-82a2-4675fe26085f admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:15.951916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e792f-813a-440a-82a2-4675fe26085f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:15.952195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e792f-813a-440a-82a2-4675fe26085f admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:15.952894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e792f-813a-440a-82a2-4675fe26085f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:15.954806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e792f-813a-440a-82a2-4675fe26085f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:16.037262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1222/2478] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:16:15 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:16:16.041027 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e95d5-9d27-4a91-adc2-8f7443761a9f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:16.051809 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e95d5-9d27-4a91-adc2-8f7443761a9f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:16.056652 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fc10f-3a26-490c-9b7a-8b55bfc92c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:16.057335 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fc10f-3a26-490c-9b7a-8b55bfc92c66 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:16.057626 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fc10f-3a26-490c-9b7a-8b55bfc92c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:16.057911 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fc10f-3a26-490c-9b7a-8b55bfc92c66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:16.058650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5fc10f-3a26-490c-9b7a-8b55bfc92c66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:16.060314 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5fc10f-3a26-490c-9b7a-8b55bfc92c66 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:16.072040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1257/2479] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:16:16.076055 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e792f-813a-440a-82a2-4675fe26085f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:16.087355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e792f-813a-440a-82a2-4675fe26085f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:16.093781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456aa989-08b4-4f6d-b0be-17525a7e8fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:16.094458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456aa989-08b4-4f6d-b0be-17525a7e8fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:16.094750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456aa989-08b4-4f6d-b0be-17525a7e8fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:16.095034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456aa989-08b4-4f6d-b0be-17525a7e8fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:16.095849 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456aa989-08b4-4f6d-b0be-17525a7e8fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:16.097457 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456aa989-08b4-4f6d-b0be-17525a7e8fb5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:16.111074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1223/2480] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:16:16.112325 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fc10f-3a26-490c-9b7a-8b55bfc92c66 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:16.122576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fc10f-3a26-490c-9b7a-8b55bfc92c66 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:16.127916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b8e1e-3a56-428c-b31e-2e3136b85e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:16.128847 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b8e1e-3a56-428c-b31e-2e3136b85e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:16.129138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b8e1e-3a56-428c-b31e-2e3136b85e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:16.129414 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b8e1e-3a56-428c-b31e-2e3136b85e61 admin admin] PATH_INFO: `/v3/projects/403729dd4efe4a78be7960a3f2477640/users/fc71eca1f2644883a0429d0c71b55d43/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:16.145795 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b8e1e-3a56-428c-b31e-2e3136b85e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=403729dd4efe4a78be7960a3f2477640, user_id=fc71eca1f2644883a0429d0c71b55d43, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:16.148410 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b8e1e-3a56-428c-b31e-2e3136b85e61 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:16.254608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1258/2481] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:16:16 2022] PUT /identity/v3/projects/403729dd4efe4a78be7960a3f2477640/users/fc71eca1f2644883a0429d0c71b55d43/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:16.265029 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9aff4-d6ed-42ff-b39f-adb3af75eefe None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:16.265325 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9aff4-d6ed-42ff-b39f-adb3af75eefe None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:16.265657 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9aff4-d6ed-42ff-b39f-adb3af75eefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:16.291779 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-30c9aff4-d6ed-42ff-b39f-adb3af75eefe None None] MFA Rules not processed for user `fc71eca1f2644883a0429d0c71b55d43`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:16.292995 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-30c9aff4-d6ed-42ff-b39f-adb3af75eefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:16.356757 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1224/2482] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:16:16.361786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b8e1e-3a56-428c-b31e-2e3136b85e61 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:16.422584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b8e1e-3a56-428c-b31e-2e3136b85e61 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:16.427854 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6dbe2-da74-4f04-a1ee-3eb5faace9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:16.429021 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6dbe2-da74-4f04-a1ee-3eb5faace9a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:16.429021 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6dbe2-da74-4f04-a1ee-3eb5faace9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:16.429241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6dbe2-da74-4f04-a1ee-3eb5faace9a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:16.429865 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d6dbe2-da74-4f04-a1ee-3eb5faace9a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:16.431818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d6dbe2-da74-4f04-a1ee-3eb5faace9a2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:16.524122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1259/2483] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:16:16 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:16:16.528320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9aff4-d6ed-42ff-b39f-adb3af75eefe None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:16.539152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9aff4-d6ed-42ff-b39f-adb3af75eefe None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:16.593628 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8532a6f-fe38-46f8-a413-a57de96a69f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:16.594295 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8532a6f-fe38-46f8-a413-a57de96a69f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:16.594581 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8532a6f-fe38-46f8-a413-a57de96a69f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:16.594859 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8532a6f-fe38-46f8-a413-a57de96a69f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:16.595586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8532a6f-fe38-46f8-a413-a57de96a69f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:16.597499 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8532a6f-fe38-46f8-a413-a57de96a69f4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:16.677905 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1225/2484] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:16:16 2022] POST /identity/v3/users => generated 300 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:16:16.681857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6dbe2-da74-4f04-a1ee-3eb5faace9a2 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:16.692139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6dbe2-da74-4f04-a1ee-3eb5faace9a2 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:16.697008 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fa91e-b50f-4ef2-8fad-82db9d0e7c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:16.702687 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fa91e-b50f-4ef2-8fad-82db9d0e7c24 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:16.702687 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fa91e-b50f-4ef2-8fad-82db9d0e7c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:16.702687 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fa91e-b50f-4ef2-8fad-82db9d0e7c24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:16.702687 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fa91e-b50f-4ef2-8fad-82db9d0e7c24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:16.702687 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fa91e-b50f-4ef2-8fad-82db9d0e7c24 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:16.711580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1260/2485] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:16:16.715630 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8532a6f-fe38-46f8-a413-a57de96a69f4 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:16.726271 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8532a6f-fe38-46f8-a413-a57de96a69f4 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:16.730847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4479142-8c72-4869-bb82-4fd4d0f9edfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:16.731832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4479142-8c72-4869-bb82-4fd4d0f9edfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:16.732118 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4479142-8c72-4869-bb82-4fd4d0f9edfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:16.732409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4479142-8c72-4869-bb82-4fd4d0f9edfa admin admin] PATH_INFO: `/v3/projects/4efe047525ea4901adeab61f8839a249/users/3af2e4d936e14859bd143c0a022c55f8/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:16.747696 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4479142-8c72-4869-bb82-4fd4d0f9edfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4efe047525ea4901adeab61f8839a249, user_id=3af2e4d936e14859bd143c0a022c55f8, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:16.750585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4479142-8c72-4869-bb82-4fd4d0f9edfa admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:16.830597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1226/2486] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:16:16 2022] PUT /identity/v3/projects/4efe047525ea4901adeab61f8839a249/users/3af2e4d936e14859bd143c0a022c55f8/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:16.834844 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fa91e-b50f-4ef2-8fad-82db9d0e7c24 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:16.844879 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fa91e-b50f-4ef2-8fad-82db9d0e7c24 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:16.865695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:16.906490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd4512-5554-4720-8350-68951a188bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:16.907159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd4512-5554-4720-8350-68951a188bee admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:16.907538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd4512-5554-4720-8350-68951a188bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:16.907819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd4512-5554-4720-8350-68951a188bee admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:16.909180 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd4512-5554-4720-8350-68951a188bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:16.910779 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd4512-5554-4720-8350-68951a188bee admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:16.920791 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1261/2487] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:16:16.925541 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4479142-8c72-4869-bb82-4fd4d0f9edfa admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:16.936253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4479142-8c72-4869-bb82-4fd4d0f9edfa admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:16.943576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00531432-bd72-404e-9a88-fc39cf9656c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:16.943576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00531432-bd72-404e-9a88-fc39cf9656c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:16.943576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00531432-bd72-404e-9a88-fc39cf9656c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:16.943576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00531432-bd72-404e-9a88-fc39cf9656c5 admin admin] PATH_INFO: `/v3/projects/4efe047525ea4901adeab61f8839a249/users/3af2e4d936e14859bd143c0a022c55f8/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:16.944724 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00531432-bd72-404e-9a88-fc39cf9656c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4efe047525ea4901adeab61f8839a249, user_id=3af2e4d936e14859bd143c0a022c55f8, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:16.947412 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00531432-bd72-404e-9a88-fc39cf9656c5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:16.958104 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1227/2488] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:16:16 2022] PUT /identity/v3/projects/4efe047525ea4901adeab61f8839a249/users/3af2e4d936e14859bd143c0a022c55f8/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:16.968683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49114934-f321-41b9-86ea-6b01ebbe0c63 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:16.972979 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49114934-f321-41b9-86ea-6b01ebbe0c63 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:16.972979 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49114934-f321-41b9-86ea-6b01ebbe0c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:17.000962 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-49114934-f321-41b9-86ea-6b01ebbe0c63 None None] MFA Rules not processed for user `3af2e4d936e14859bd143c0a022c55f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:17.002321 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-49114934-f321-41b9-86ea-6b01ebbe0c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:17.073422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1262/2489] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:16:17.085863 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00531432-bd72-404e-9a88-fc39cf9656c5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:17.087511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-00531432-bd72-404e-9a88-fc39cf9656c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:17.111001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:17.172007 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00531432-bd72-404e-9a88-fc39cf9656c5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:17.175830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d6cb5-3351-4f48-8708-f365d66c06aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:17.176535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d6cb5-3351-4f48-8708-f365d66c06aa service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:17.176824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d6cb5-3351-4f48-8708-f365d66c06aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:17.177103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d6cb5-3351-4f48-8708-f365d66c06aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:17.187123 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d6cb5-3351-4f48-8708-f365d66c06aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:17.188743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d6cb5-3351-4f48-8708-f365d66c06aa service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:17.200799 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1228/2490] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:16:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:16:17.271526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49114934-f321-41b9-86ea-6b01ebbe0c63 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:17.338976 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49114934-f321-41b9-86ea-6b01ebbe0c63 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:17.343714 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc7eae-2f0b-4a39-be76-8f631334a0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:17.344590 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc7eae-2f0b-4a39-be76-8f631334a0c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:17.344884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc7eae-2f0b-4a39-be76-8f631334a0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:17.345257 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc7eae-2f0b-4a39-be76-8f631334a0c8 admin admin] PATH_INFO: `/v3/users/fc71eca1f2644883a0429d0c71b55d43` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:17.346990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc7eae-2f0b-4a39-be76-8f631334a0c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc71eca1f2644883a0429d0c71b55d43)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:17.348946 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc7eae-2f0b-4a39-be76-8f631334a0c8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:17.508116 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-03bc7eae-2f0b-4a39-be76-8f631334a0c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc71eca1f2644883a0429d0c71b55d43'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:17.508778 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-03bc7eae-2f0b-4a39-be76-8f631334a0c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc71eca1f2644883a0429d0c71b55d43'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:17.509947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:17.522217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-03bc7eae-2f0b-4a39-be76-8f631334a0c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc71eca1f2644883a0429d0c71b55d43'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:17.522917 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-03bc7eae-2f0b-4a39-be76-8f631334a0c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc71eca1f2644883a0429d0c71b55d43'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:17.530206 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-03bc7eae-2f0b-4a39-be76-8f631334a0c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc71eca1f2644883a0429d0c71b55d43'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:17.597357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1263/2491] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:16:17 2022] DELETE /identity/v3/users/fc71eca1f2644883a0429d0c71b55d43 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:17.601911 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe60d1-c508-4164-893b-d62345d6c478 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:17.602209 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe60d1-c508-4164-893b-d62345d6c478 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:17.602517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe60d1-c508-4164-893b-d62345d6c478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:17.619648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-97fe60d1-c508-4164-893b-d62345d6c478 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:17.620943 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-97fe60d1-c508-4164-893b-d62345d6c478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:17.680790 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1229/2492] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:16:17.691648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc7eae-2f0b-4a39-be76-8f631334a0c8 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:17.693136 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-03bc7eae-2f0b-4a39-be76-8f631334a0c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:17.750050 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc7eae-2f0b-4a39-be76-8f631334a0c8 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:17.754284 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12500a2f-e50d-4ff7-89db-8645df4421b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:17.754930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12500a2f-e50d-4ff7-89db-8645df4421b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:17.755242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12500a2f-e50d-4ff7-89db-8645df4421b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:17.755521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12500a2f-e50d-4ff7-89db-8645df4421b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:17.764357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12500a2f-e50d-4ff7-89db-8645df4421b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:17.765840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12500a2f-e50d-4ff7-89db-8645df4421b3 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:17.775310 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1264/2493] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:16:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:16:18.148131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe60d1-c508-4164-893b-d62345d6c478 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:18.149634 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-97fe60d1-c508-4164-893b-d62345d6c478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:18.169199 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe60d1-c508-4164-893b-d62345d6c478 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:18.174751 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3e01b-793d-4026-a4e2-146d63f5b5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:18.175680 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3e01b-793d-4026-a4e2-146d63f5b5c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:18.175973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3e01b-793d-4026-a4e2-146d63f5b5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:18.176301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3e01b-793d-4026-a4e2-146d63f5b5c7 admin admin] PATH_INFO: `/v3/projects/403729dd4efe4a78be7960a3f2477640` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:18.177427 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3e01b-793d-4026-a4e2-146d63f5b5c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=403729dd4efe4a78be7960a3f2477640)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:18.179454 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3e01b-793d-4026-a4e2-146d63f5b5c7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:18.271671 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-19d3e01b-793d-4026-a4e2-146d63f5b5c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '403729dd4efe4a78be7960a3f2477640', 'request_id': 'req-19d3e01b-793d-4026-a4e2-146d63f5b5c7'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:18.339706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-19d3e01b-793d-4026-a4e2-146d63f5b5c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '403729dd4efe4a78be7960a3f2477640', 'request_id': 'req-19d3e01b-793d-4026-a4e2-146d63f5b5c7'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:18.346198 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-19d3e01b-793d-4026-a4e2-146d63f5b5c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '403729dd4efe4a78be7960a3f2477640', 'request_id': 'req-19d3e01b-793d-4026-a4e2-146d63f5b5c7'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:18.357321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-19d3e01b-793d-4026-a4e2-146d63f5b5c7 admin admin] The token cache is being invalidate because project 403729dd4efe4a78be7960a3f2477640 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:16:18.357611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-19d3e01b-793d-4026-a4e2-146d63f5b5c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:18.358247 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-19d3e01b-793d-4026-a4e2-146d63f5b5c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:18.359831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1230/2494] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:16:18 2022] DELETE /identity/v3/projects/403729dd4efe4a78be7960a3f2477640 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:18.364339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12500a2f-e50d-4ff7-89db-8645df4421b3 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:18.374106 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12500a2f-e50d-4ff7-89db-8645df4421b3 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:18.434421 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665e1ee-13bb-48ba-8b94-0f90aa80a302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:18.435239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665e1ee-13bb-48ba-8b94-0f90aa80a302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:18.435532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665e1ee-13bb-48ba-8b94-0f90aa80a302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:18.435813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665e1ee-13bb-48ba-8b94-0f90aa80a302 admin admin] PATH_INFO: `/v3/users/3af2e4d936e14859bd143c0a022c55f8` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:18.437005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665e1ee-13bb-48ba-8b94-0f90aa80a302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3af2e4d936e14859bd143c0a022c55f8)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:18.438955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665e1ee-13bb-48ba-8b94-0f90aa80a302 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:18.576205 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1665e1ee-13bb-48ba-8b94-0f90aa80a302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3af2e4d936e14859bd143c0a022c55f8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:18.576835 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1665e1ee-13bb-48ba-8b94-0f90aa80a302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3af2e4d936e14859bd143c0a022c55f8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:18.589124 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1665e1ee-13bb-48ba-8b94-0f90aa80a302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3af2e4d936e14859bd143c0a022c55f8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:18.589802 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1665e1ee-13bb-48ba-8b94-0f90aa80a302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3af2e4d936e14859bd143c0a022c55f8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:18.594379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:18.597793 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1665e1ee-13bb-48ba-8b94-0f90aa80a302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3af2e4d936e14859bd143c0a022c55f8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:18.670035 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1265/2495] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:16:18 2022] DELETE /identity/v3/users/3af2e4d936e14859bd143c0a022c55f8 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:19.061519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3e01b-793d-4026-a4e2-146d63f5b5c7 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:19.063125 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-19d3e01b-793d-4026-a4e2-146d63f5b5c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:19.077194 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3e01b-793d-4026-a4e2-146d63f5b5c7 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:19.098728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:19.134071 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6febd7-f4b0-455b-b7d4-07d7c56faadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:19.135088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6febd7-f4b0-455b-b7d4-07d7c56faadc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:19.135757 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6febd7-f4b0-455b-b7d4-07d7c56faadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:19.136087 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6febd7-f4b0-455b-b7d4-07d7c56faadc admin admin] PATH_INFO: `/v3/projects/4efe047525ea4901adeab61f8839a249` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:19.137273 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6febd7-f4b0-455b-b7d4-07d7c56faadc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4efe047525ea4901adeab61f8839a249)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:19.139211 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6febd7-f4b0-455b-b7d4-07d7c56faadc admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:19.242369 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9c6febd7-f4b0-455b-b7d4-07d7c56faadc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4efe047525ea4901adeab61f8839a249', 'request_id': 'req-9c6febd7-f4b0-455b-b7d4-07d7c56faadc'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:19.294015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9c6febd7-f4b0-455b-b7d4-07d7c56faadc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4efe047525ea4901adeab61f8839a249', 'request_id': 'req-9c6febd7-f4b0-455b-b7d4-07d7c56faadc'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:19.300359 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9c6febd7-f4b0-455b-b7d4-07d7c56faadc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4efe047525ea4901adeab61f8839a249', 'request_id': 'req-9c6febd7-f4b0-455b-b7d4-07d7c56faadc'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:19.311496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9c6febd7-f4b0-455b-b7d4-07d7c56faadc admin admin] The token cache is being invalidate because project 4efe047525ea4901adeab61f8839a249 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:16:19.311856 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9c6febd7-f4b0-455b-b7d4-07d7c56faadc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:19.312548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9c6febd7-f4b0-455b-b7d4-07d7c56faadc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:19.314300 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1231/2496] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:16:19 2022] DELETE /identity/v3/projects/4efe047525ea4901adeab61f8839a249 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:19.319676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca8ece-cc0f-486f-aa9a-ae4d70d8273e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:19.319964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca8ece-cc0f-486f-aa9a-ae4d70d8273e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:19.320264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca8ece-cc0f-486f-aa9a-ae4d70d8273e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:19.338496 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3eca8ece-cc0f-486f-aa9a-ae4d70d8273e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:19.339720 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3eca8ece-cc0f-486f-aa9a-ae4d70d8273e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:19.410578 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1266/2497] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:16:19.419249 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc3a64-bcce-4bf3-92e8-e88ab12081bf None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:19.419550 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc3a64-bcce-4bf3-92e8-e88ab12081bf None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:19.419832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc3a64-bcce-4bf3-92e8-e88ab12081bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:19.439080 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-78bc3a64-bcce-4bf3-92e8-e88ab12081bf None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:19.441075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-78bc3a64-bcce-4bf3-92e8-e88ab12081bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:19.449038 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1232/2498] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:16:19.453384 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca8ece-cc0f-486f-aa9a-ae4d70d8273e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:19.463862 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca8ece-cc0f-486f-aa9a-ae4d70d8273e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:19.468452 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f2c82-39b6-4aec-9239-55a9ac48035a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:19.469136 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f2c82-39b6-4aec-9239-55a9ac48035a admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:19.469418 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f2c82-39b6-4aec-9239-55a9ac48035a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:19.469695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f2c82-39b6-4aec-9239-55a9ac48035a admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:19.470471 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f2c82-39b6-4aec-9239-55a9ac48035a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:16:19.470805 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f2c82-39b6-4aec-9239-55a9ac48035a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:19.472454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f2c82-39b6-4aec-9239-55a9ac48035a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:19.484378 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1267/2499] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:16:19.489418 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc3a64-bcce-4bf3-92e8-e88ab12081bf None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:19.498978 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc3a64-bcce-4bf3-92e8-e88ab12081bf None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:19.504545 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb21663-4ccb-416c-bcee-acd0684a2a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:19.505221 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb21663-4ccb-416c-bcee-acd0684a2a2e admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:19.505495 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb21663-4ccb-416c-bcee-acd0684a2a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:19.505771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb21663-4ccb-416c-bcee-acd0684a2a2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:19.506464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb21663-4ccb-416c-bcee-acd0684a2a2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:19.508454 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb21663-4ccb-416c-bcee-acd0684a2a2e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:19.604857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1233/2500] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:16:19 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:16:19.608932 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f2c82-39b6-4aec-9239-55a9ac48035a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:19.619667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f2c82-39b6-4aec-9239-55a9ac48035a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:19.684319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed5bd3-28b8-4d00-a1f9-73a5fa31c32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:19.685035 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed5bd3-28b8-4d00-a1f9-73a5fa31c32f admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:19.685326 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed5bd3-28b8-4d00-a1f9-73a5fa31c32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:19.685604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed5bd3-28b8-4d00-a1f9-73a5fa31c32f admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:19.686312 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed5bd3-28b8-4d00-a1f9-73a5fa31c32f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:19.688466 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed5bd3-28b8-4d00-a1f9-73a5fa31c32f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:19.769015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1268/2501] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:16:19 2022] POST /identity/v3/users => generated 283 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:16:19.773236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb21663-4ccb-416c-bcee-acd0684a2a2e admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:19.784379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb21663-4ccb-416c-bcee-acd0684a2a2e admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:19.788984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d01abf-c9f5-4a9f-ae52-d8b77eb4a2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:19.794637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d01abf-c9f5-4a9f-ae52-d8b77eb4a2d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:19.794637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d01abf-c9f5-4a9f-ae52-d8b77eb4a2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:19.794637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d01abf-c9f5-4a9f-ae52-d8b77eb4a2d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:19.794637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d01abf-c9f5-4a9f-ae52-d8b77eb4a2d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:19.794637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d01abf-c9f5-4a9f-ae52-d8b77eb4a2d7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:19.804097 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1234/2502] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:16:19.808032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed5bd3-28b8-4d00-a1f9-73a5fa31c32f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:19.818057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed5bd3-28b8-4d00-a1f9-73a5fa31c32f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:19.822636 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166354f8-c483-4d87-bb8a-7a376b62c40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:19.823323 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166354f8-c483-4d87-bb8a-7a376b62c40c admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:19.823612 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166354f8-c483-4d87-bb8a-7a376b62c40c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:19.823890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166354f8-c483-4d87-bb8a-7a376b62c40c admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:19.824607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166354f8-c483-4d87-bb8a-7a376b62c40c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:19.826197 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166354f8-c483-4d87-bb8a-7a376b62c40c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:19.836133 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1269/2503] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:16:19.843557 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d01abf-c9f5-4a9f-ae52-d8b77eb4a2d7 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:19.854155 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d01abf-c9f5-4a9f-ae52-d8b77eb4a2d7 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:19.858747 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e7c5e-1e04-42b6-b52b-1ecd3580f58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:19.859648 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e7c5e-1e04-42b6-b52b-1ecd3580f58c admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:19.859932 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e7c5e-1e04-42b6-b52b-1ecd3580f58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:19.860206 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e7c5e-1e04-42b6-b52b-1ecd3580f58c admin admin] PATH_INFO: `/v3/projects/9b7d8f9b89fb4b7bb78de750b030f840/users/05457832b6644d87a7ec39dfdafcbb23/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:19.875120 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e7c5e-1e04-42b6-b52b-1ecd3580f58c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b7d8f9b89fb4b7bb78de750b030f840, user_id=05457832b6644d87a7ec39dfdafcbb23, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:19.877774 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e7c5e-1e04-42b6-b52b-1ecd3580f58c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:19.969188 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1235/2504] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:16:19 2022] PUT /identity/v3/projects/9b7d8f9b89fb4b7bb78de750b030f840/users/05457832b6644d87a7ec39dfdafcbb23/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:16:19.980055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058dd52-09f2-449f-8635-98f44ba203e9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:19.980469 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058dd52-09f2-449f-8635-98f44ba203e9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:19.980843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058dd52-09f2-449f-8635-98f44ba203e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:20.008486 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e058dd52-09f2-449f-8635-98f44ba203e9 None None] MFA Rules not processed for user `05457832b6644d87a7ec39dfdafcbb23`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:20.009694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e058dd52-09f2-449f-8635-98f44ba203e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:20.050953 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:20.073002 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1270/2505] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:19 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:16:20.077528 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e7c5e-1e04-42b6-b52b-1ecd3580f58c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:20.131722 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:20.144582 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e7c5e-1e04-42b6-b52b-1ecd3580f58c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:20.149966 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27968578-9657-4f1e-ad49-b64fb11b663b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:20.150906 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27968578-9657-4f1e-ad49-b64fb11b663b admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:20.151225 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27968578-9657-4f1e-ad49-b64fb11b663b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:20.151507 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27968578-9657-4f1e-ad49-b64fb11b663b admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:20.152217 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27968578-9657-4f1e-ad49-b64fb11b663b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:20.154163 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27968578-9657-4f1e-ad49-b64fb11b663b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:20.235985 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1236/2506] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:16:20 2022] POST /identity/v3/projects => generated 367 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:16:20.238335 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058dd52-09f2-449f-8635-98f44ba203e9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:20.249656 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058dd52-09f2-449f-8635-98f44ba203e9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:20.305979 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d706c-2fa7-4b1c-8753-4f6a63115d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:20.306576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d706c-2fa7-4b1c-8753-4f6a63115d0f admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:20.306868 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d706c-2fa7-4b1c-8753-4f6a63115d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:20.307181 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d706c-2fa7-4b1c-8753-4f6a63115d0f admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:20.307870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d706c-2fa7-4b1c-8753-4f6a63115d0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:20.309802 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d706c-2fa7-4b1c-8753-4f6a63115d0f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:20.363407 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1271/2507] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:16:20 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:16:20.367633 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27968578-9657-4f1e-ad49-b64fb11b663b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:20.378512 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27968578-9657-4f1e-ad49-b64fb11b663b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:20.383113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5393f-f665-4115-83e5-03efb2af6e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:20.383799 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5393f-f665-4115-83e5-03efb2af6e9f admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:20.384084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5393f-f665-4115-83e5-03efb2af6e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:20.384362 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5393f-f665-4115-83e5-03efb2af6e9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:20.385079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5393f-f665-4115-83e5-03efb2af6e9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:20.386675 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5393f-f665-4115-83e5-03efb2af6e9f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:20.397050 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1237/2508] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:16:20.401552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d706c-2fa7-4b1c-8753-4f6a63115d0f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:20.412193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d706c-2fa7-4b1c-8753-4f6a63115d0f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:20.417171 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd01179-5d74-4f97-9c21-ec9bbb5a5c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:20.418048 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd01179-5d74-4f97-9c21-ec9bbb5a5c2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:20.418338 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd01179-5d74-4f97-9c21-ec9bbb5a5c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:20.418653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd01179-5d74-4f97-9c21-ec9bbb5a5c2d admin admin] PATH_INFO: `/v3/projects/265fc737e5064683b5c1ed0ecf0ff163/users/674572ff74164c5889a45c11b314f9cb/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:20.434246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd01179-5d74-4f97-9c21-ec9bbb5a5c2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=265fc737e5064683b5c1ed0ecf0ff163, user_id=674572ff74164c5889a45c11b314f9cb, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:20.436951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd01179-5d74-4f97-9c21-ec9bbb5a5c2d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:20.514364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1272/2509] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:16:20 2022] PUT /identity/v3/projects/265fc737e5064683b5c1ed0ecf0ff163/users/674572ff74164c5889a45c11b314f9cb/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:20.518460 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5393f-f665-4115-83e5-03efb2af6e9f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:20.529338 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5393f-f665-4115-83e5-03efb2af6e9f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:20.585836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6075a2-93ff-474c-b20c-e131cfb0a01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:20.586507 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6075a2-93ff-474c-b20c-e131cfb0a01b admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:20.586825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6075a2-93ff-474c-b20c-e131cfb0a01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:20.587108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6075a2-93ff-474c-b20c-e131cfb0a01b admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:20.587859 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6075a2-93ff-474c-b20c-e131cfb0a01b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:20.589462 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6075a2-93ff-474c-b20c-e131cfb0a01b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:20.599818 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1238/2510] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:16:20.604029 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd01179-5d74-4f97-9c21-ec9bbb5a5c2d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:20.615104 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd01179-5d74-4f97-9c21-ec9bbb5a5c2d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:20.619836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a97200-a71e-44d0-bbc0-e2e92bbb822c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:20.620726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a97200-a71e-44d0-bbc0-e2e92bbb822c admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:20.621010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a97200-a71e-44d0-bbc0-e2e92bbb822c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:20.621293 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a97200-a71e-44d0-bbc0-e2e92bbb822c admin admin] PATH_INFO: `/v3/projects/265fc737e5064683b5c1ed0ecf0ff163/users/674572ff74164c5889a45c11b314f9cb/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:20.623317 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a97200-a71e-44d0-bbc0-e2e92bbb822c admin admin] RBAC: Authorizing `identity:create_grant(project_id=265fc737e5064683b5c1ed0ecf0ff163, user_id=674572ff74164c5889a45c11b314f9cb, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:20.625961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a97200-a71e-44d0-bbc0-e2e92bbb822c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:20.636391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1273/2511] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:16:20 2022] PUT /identity/v3/projects/265fc737e5064683b5c1ed0ecf0ff163/users/674572ff74164c5889a45c11b314f9cb/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:20.647791 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a99a9-59a2-4c90-af32-a75fa822f8b5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:20.648086 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a99a9-59a2-4c90-af32-a75fa822f8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:20.648369 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a99a9-59a2-4c90-af32-a75fa822f8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:20.675846 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-685a99a9-59a2-4c90-af32-a75fa822f8b5 None None] MFA Rules not processed for user `674572ff74164c5889a45c11b314f9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:20.677109 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-685a99a9-59a2-4c90-af32-a75fa822f8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:20.745167 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1239/2512] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:20 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:16:20.756870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a97200-a71e-44d0-bbc0-e2e92bbb822c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:20.758577 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a1a97200-a71e-44d0-bbc0-e2e92bbb822c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:20.816598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a97200-a71e-44d0-bbc0-e2e92bbb822c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:20.820499 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76831161-33f9-4304-bb65-ab7ca1c1e931 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:20.821157 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76831161-33f9-4304-bb65-ab7ca1c1e931 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:20.821448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76831161-33f9-4304-bb65-ab7ca1c1e931 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:20.821732 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76831161-33f9-4304-bb65-ab7ca1c1e931 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:20.832286 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76831161-33f9-4304-bb65-ab7ca1c1e931 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:20.836955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76831161-33f9-4304-bb65-ab7ca1c1e931 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:20.849264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1274/2513] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:16:20 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:16:21.079371 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a99a9-59a2-4c90-af32-a75fa822f8b5 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:21.089580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a99a9-59a2-4c90-af32-a75fa822f8b5 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:21.093302 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6779ba2c-2c39-4eb8-b6df-462524eb616c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:21.093974 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6779ba2c-2c39-4eb8-b6df-462524eb616c service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:21.094268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6779ba2c-2c39-4eb8-b6df-462524eb616c service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:21.097689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6779ba2c-2c39-4eb8-b6df-462524eb616c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:21.140183 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:21.153230 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6779ba2c-2c39-4eb8-b6df-462524eb616c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:21.154803 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6779ba2c-2c39-4eb8-b6df-462524eb616c service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:21.163709 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1240/2514] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:16:21 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:16:28.474527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76831161-33f9-4304-bb65-ab7ca1c1e931 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:28.514648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:28.537273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76831161-33f9-4304-bb65-ab7ca1c1e931 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:28.541809 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d48e8-f820-48b7-b26c-89f55564dbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:28.542676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d48e8-f820-48b7-b26c-89f55564dbb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:28.542966 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d48e8-f820-48b7-b26c-89f55564dbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:28.543277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d48e8-f820-48b7-b26c-89f55564dbb6 admin admin] PATH_INFO: `/v3/users/05457832b6644d87a7ec39dfdafcbb23` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:28.544517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d48e8-f820-48b7-b26c-89f55564dbb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05457832b6644d87a7ec39dfdafcbb23)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:28.547277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d48e8-f820-48b7-b26c-89f55564dbb6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:28.754023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d25d48e8-f820-48b7-b26c-89f55564dbb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05457832b6644d87a7ec39dfdafcbb23'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:28.754585 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d25d48e8-f820-48b7-b26c-89f55564dbb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05457832b6644d87a7ec39dfdafcbb23'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:28.768153 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d25d48e8-f820-48b7-b26c-89f55564dbb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05457832b6644d87a7ec39dfdafcbb23'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:28.768804 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d25d48e8-f820-48b7-b26c-89f55564dbb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05457832b6644d87a7ec39dfdafcbb23'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:28.776016 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d25d48e8-f820-48b7-b26c-89f55564dbb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05457832b6644d87a7ec39dfdafcbb23'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:28.837165 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1275/2515] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:16:28 2022] DELETE /identity/v3/users/05457832b6644d87a7ec39dfdafcbb23 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:28.842982 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2678f-0519-4c39-9f81-d325847db319 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:28.842982 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2678f-0519-4c39-9f81-d325847db319 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:28.842982 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2678f-0519-4c39-9f81-d325847db319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:28.860386 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-5ce2678f-0519-4c39-9f81-d325847db319 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:28.861609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5ce2678f-0519-4c39-9f81-d325847db319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:28.920007 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1241/2516] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:16:28.929604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d48e8-f820-48b7-b26c-89f55564dbb6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:28.931035 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d25d48e8-f820-48b7-b26c-89f55564dbb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:28.988301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d48e8-f820-48b7-b26c-89f55564dbb6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:28.991897 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fadce-695e-4a19-8265-ad0ba1ec0df3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:28.992566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fadce-695e-4a19-8265-ad0ba1ec0df3 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:28.992932 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fadce-695e-4a19-8265-ad0ba1ec0df3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:28.993218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fadce-695e-4a19-8265-ad0ba1ec0df3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:29.003239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fadce-695e-4a19-8265-ad0ba1ec0df3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:29.004797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fadce-695e-4a19-8265-ad0ba1ec0df3 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:29.014524 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1276/2517] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:16:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:16:29.444171 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2678f-0519-4c39-9f81-d325847db319 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:29.445617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5ce2678f-0519-4c39-9f81-d325847db319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:29.457855 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2678f-0519-4c39-9f81-d325847db319 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:29.463059 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae75c01-fa90-4a96-9ab7-d01db10caceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:29.463967 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae75c01-fa90-4a96-9ab7-d01db10caceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:29.464444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae75c01-fa90-4a96-9ab7-d01db10caceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:29.464729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae75c01-fa90-4a96-9ab7-d01db10caceb admin admin] PATH_INFO: `/v3/projects/9b7d8f9b89fb4b7bb78de750b030f840` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:29.465959 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae75c01-fa90-4a96-9ab7-d01db10caceb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b7d8f9b89fb4b7bb78de750b030f840)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:29.469898 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae75c01-fa90-4a96-9ab7-d01db10caceb admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:29.574192 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-eae75c01-fa90-4a96-9ab7-d01db10caceb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b7d8f9b89fb4b7bb78de750b030f840', 'request_id': 'req-eae75c01-fa90-4a96-9ab7-d01db10caceb'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:29.631174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-eae75c01-fa90-4a96-9ab7-d01db10caceb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b7d8f9b89fb4b7bb78de750b030f840', 'request_id': 'req-eae75c01-fa90-4a96-9ab7-d01db10caceb'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:29.637124 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-eae75c01-fa90-4a96-9ab7-d01db10caceb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b7d8f9b89fb4b7bb78de750b030f840', 'request_id': 'req-eae75c01-fa90-4a96-9ab7-d01db10caceb'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:29.648167 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-eae75c01-fa90-4a96-9ab7-d01db10caceb admin admin] The token cache is being invalidate because project 9b7d8f9b89fb4b7bb78de750b030f840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:16:29.648457 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-eae75c01-fa90-4a96-9ab7-d01db10caceb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:29.649141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-eae75c01-fa90-4a96-9ab7-d01db10caceb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:29.651202 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1242/2518] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:16:29 2022] DELETE /identity/v3/projects/9b7d8f9b89fb4b7bb78de750b030f840 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:29.654948 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fadce-695e-4a19-8265-ad0ba1ec0df3 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:29.665364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fadce-695e-4a19-8265-ad0ba1ec0df3 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:29.720937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5912b-e541-468d-8da9-11b3a188ff68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:29.722428 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5912b-e541-468d-8da9-11b3a188ff68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:29.722715 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5912b-e541-468d-8da9-11b3a188ff68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:29.722994 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5912b-e541-468d-8da9-11b3a188ff68 admin admin] PATH_INFO: `/v3/users/674572ff74164c5889a45c11b314f9cb` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:29.724221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5912b-e541-468d-8da9-11b3a188ff68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=674572ff74164c5889a45c11b314f9cb)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:29.726161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5912b-e541-468d-8da9-11b3a188ff68 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:29.727642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:29.868013 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c0e5912b-e541-468d-8da9-11b3a188ff68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '674572ff74164c5889a45c11b314f9cb'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:29.868576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c0e5912b-e541-468d-8da9-11b3a188ff68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '674572ff74164c5889a45c11b314f9cb'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:29.880736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c0e5912b-e541-468d-8da9-11b3a188ff68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '674572ff74164c5889a45c11b314f9cb'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:29.881937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c0e5912b-e541-468d-8da9-11b3a188ff68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '674572ff74164c5889a45c11b314f9cb'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:29.888478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c0e5912b-e541-468d-8da9-11b3a188ff68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '674572ff74164c5889a45c11b314f9cb'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:29.967695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1277/2519] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:16:29 2022] DELETE /identity/v3/users/674572ff74164c5889a45c11b314f9cb => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:30.398988 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae75c01-fa90-4a96-9ab7-d01db10caceb admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:30.400568 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-eae75c01-fa90-4a96-9ab7-d01db10caceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:30.414193 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae75c01-fa90-4a96-9ab7-d01db10caceb admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:30.469929 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b4450-62f0-41ac-a67d-ceddeb9fe501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:30.470784 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b4450-62f0-41ac-a67d-ceddeb9fe501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:30.471066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b4450-62f0-41ac-a67d-ceddeb9fe501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:30.471366 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b4450-62f0-41ac-a67d-ceddeb9fe501 admin admin] PATH_INFO: `/v3/projects/265fc737e5064683b5c1ed0ecf0ff163` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:30.472561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b4450-62f0-41ac-a67d-ceddeb9fe501 admin admin] RBAC: Authorizing `identity:delete_project(project_id=265fc737e5064683b5c1ed0ecf0ff163)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:30.474449 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b4450-62f0-41ac-a67d-ceddeb9fe501 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:30.484547 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:30.581135 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-081b4450-62f0-41ac-a67d-ceddeb9fe501 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '265fc737e5064683b5c1ed0ecf0ff163', 'request_id': 'req-081b4450-62f0-41ac-a67d-ceddeb9fe501'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:30.619798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-081b4450-62f0-41ac-a67d-ceddeb9fe501 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '265fc737e5064683b5c1ed0ecf0ff163', 'request_id': 'req-081b4450-62f0-41ac-a67d-ceddeb9fe501'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:30.625397 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-081b4450-62f0-41ac-a67d-ceddeb9fe501 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '265fc737e5064683b5c1ed0ecf0ff163', 'request_id': 'req-081b4450-62f0-41ac-a67d-ceddeb9fe501'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:30.636955 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-081b4450-62f0-41ac-a67d-ceddeb9fe501 admin admin] The token cache is being invalidate because project 265fc737e5064683b5c1ed0ecf0ff163 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:16:30.637238 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-081b4450-62f0-41ac-a67d-ceddeb9fe501 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:30.637863 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-081b4450-62f0-41ac-a67d-ceddeb9fe501 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:16:30.639398 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1243/2520] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:16:30 2022] DELETE /identity/v3/projects/265fc737e5064683b5c1ed0ecf0ff163 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:16:30.644203 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab24950-3a64-4020-94f0-ff76701a4756 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:30.644439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab24950-3a64-4020-94f0-ff76701a4756 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:30.644717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab24950-3a64-4020-94f0-ff76701a4756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:30.663385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5ab24950-3a64-4020-94f0-ff76701a4756 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:30.664582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5ab24950-3a64-4020-94f0-ff76701a4756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:30.723470 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1278/2521] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:16:30.731616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a701d-bd4f-4760-b8ef-a886b7e3dd4d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:30.731914 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a701d-bd4f-4760-b8ef-a886b7e3dd4d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:30.732201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a701d-bd4f-4760-b8ef-a886b7e3dd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:30.751008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-308a701d-bd4f-4760-b8ef-a886b7e3dd4d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:30.752224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-308a701d-bd4f-4760-b8ef-a886b7e3dd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:30.760832 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1244/2522] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:16:30.765017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab24950-3a64-4020-94f0-ff76701a4756 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:30.774631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab24950-3a64-4020-94f0-ff76701a4756 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:30.779447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6c1cf-47fb-4aae-94af-ec78a1ab5dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:30.780092 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6c1cf-47fb-4aae-94af-ec78a1ab5dec admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:30.780395 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6c1cf-47fb-4aae-94af-ec78a1ab5dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:30.780654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6c1cf-47fb-4aae-94af-ec78a1ab5dec admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:30.781370 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6c1cf-47fb-4aae-94af-ec78a1ab5dec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:16:30.781673 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6c1cf-47fb-4aae-94af-ec78a1ab5dec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:30.783332 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6c1cf-47fb-4aae-94af-ec78a1ab5dec admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:30.794601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1279/2523] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:16:30.798857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a701d-bd4f-4760-b8ef-a886b7e3dd4d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:30.808586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a701d-bd4f-4760-b8ef-a886b7e3dd4d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:30.813266 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8410db-b822-478c-930a-2f8ad202b503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:30.813939 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8410db-b822-478c-930a-2f8ad202b503 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:30.814233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8410db-b822-478c-930a-2f8ad202b503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:30.814510 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8410db-b822-478c-930a-2f8ad202b503 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:30.815238 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8410db-b822-478c-930a-2f8ad202b503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:30.817762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8410db-b822-478c-930a-2f8ad202b503 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:30.913909 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1245/2524] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:16:30 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:16:30.917914 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6c1cf-47fb-4aae-94af-ec78a1ab5dec admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:30.928493 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6c1cf-47fb-4aae-94af-ec78a1ab5dec admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:30.983483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372f3c1-b21d-4b83-b14d-577d1bb08845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:30.984143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372f3c1-b21d-4b83-b14d-577d1bb08845 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:30.984445 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372f3c1-b21d-4b83-b14d-577d1bb08845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:30.984703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372f3c1-b21d-4b83-b14d-577d1bb08845 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:30.985378 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0372f3c1-b21d-4b83-b14d-577d1bb08845 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:30.987319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0372f3c1-b21d-4b83-b14d-577d1bb08845 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:30.996970 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:31.066088 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1280/2525] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:16:30 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:16:31.070303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8410db-b822-478c-930a-2f8ad202b503 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:31.080504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8410db-b822-478c-930a-2f8ad202b503 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:31.085677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637ae0c-b6b3-4a3f-b9a8-fe26e8ec78b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:31.086317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637ae0c-b6b3-4a3f-b9a8-fe26e8ec78b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:31.086601 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637ae0c-b6b3-4a3f-b9a8-fe26e8ec78b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:31.086877 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637ae0c-b6b3-4a3f-b9a8-fe26e8ec78b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:31.087627 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637ae0c-b6b3-4a3f-b9a8-fe26e8ec78b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:31.089223 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637ae0c-b6b3-4a3f-b9a8-fe26e8ec78b9 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:31.099523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1246/2526] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:16:31.103506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372f3c1-b21d-4b83-b14d-577d1bb08845 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:31.113465 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372f3c1-b21d-4b83-b14d-577d1bb08845 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:31.118294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eacd87-e07c-4ba0-afef-2ca36bc5a6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:31.118940 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eacd87-e07c-4ba0-afef-2ca36bc5a6fd admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:31.119259 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eacd87-e07c-4ba0-afef-2ca36bc5a6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:31.119535 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eacd87-e07c-4ba0-afef-2ca36bc5a6fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:31.120250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eacd87-e07c-4ba0-afef-2ca36bc5a6fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:31.121874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eacd87-e07c-4ba0-afef-2ca36bc5a6fd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:31.133928 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1281/2527] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:16:31.137908 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637ae0c-b6b3-4a3f-b9a8-fe26e8ec78b9 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:31.148774 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637ae0c-b6b3-4a3f-b9a8-fe26e8ec78b9 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:31.153584 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814fcb8-c1bb-44d2-903d-56e5da0f2eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:31.154461 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814fcb8-c1bb-44d2-903d-56e5da0f2eda admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:31.154740 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814fcb8-c1bb-44d2-903d-56e5da0f2eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:31.155011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814fcb8-c1bb-44d2-903d-56e5da0f2eda admin admin] PATH_INFO: `/v3/projects/b21daf298af44b17bd0b2b0fc92d10f5/users/436b828f7aa04caba9083436538a8ffc/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:31.169227 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814fcb8-c1bb-44d2-903d-56e5da0f2eda admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21daf298af44b17bd0b2b0fc92d10f5, user_id=436b828f7aa04caba9083436538a8ffc, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:31.171840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814fcb8-c1bb-44d2-903d-56e5da0f2eda admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:31.219494 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1247/2528] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:16:31 2022] PUT /identity/v3/projects/b21daf298af44b17bd0b2b0fc92d10f5/users/436b828f7aa04caba9083436538a8ffc/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:16:31.229686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e93e7a-50ca-499e-9594-73a67f90b4e0 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:31.229965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e93e7a-50ca-499e-9594-73a67f90b4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:31.230255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e93e7a-50ca-499e-9594-73a67f90b4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:31.257755 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-78e93e7a-50ca-499e-9594-73a67f90b4e0 None None] MFA Rules not processed for user `436b828f7aa04caba9083436538a8ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:16:31.258971 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-78e93e7a-50ca-499e-9594-73a67f90b4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:31.323951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1282/2529] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:16:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:16:31.335034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814fcb8-c1bb-44d2-903d-56e5da0f2eda admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:16:31.336667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e814fcb8-c1bb-44d2-903d-56e5da0f2eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:16:31.393051 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814fcb8-c1bb-44d2-903d-56e5da0f2eda admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:16:31.396913 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ade672-c612-4e56-8946-f45cfab41292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:16:31.397566 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ade672-c612-4e56-8946-f45cfab41292 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:16:31.397854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ade672-c612-4e56-8946-f45cfab41292 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:16:31.398134 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ade672-c612-4e56-8946-f45cfab41292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:16:31.406739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ade672-c612-4e56-8946-f45cfab41292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:16:31.408238 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ade672-c612-4e56-8946-f45cfab41292 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:16:31.417080 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1248/2530] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:16:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:16:41.321154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=39 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:16:41.413014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:17:01.338985 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e93e7a-50ca-499e-9594-73a67f90b4e0 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:17:01.402304 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e93e7a-50ca-499e-9594-73a67f90b4e0 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:17:01.406881 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47841e6a-8368-4d61-b20e-91888b74cd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:17:01.407796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47841e6a-8368-4d61-b20e-91888b74cd56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:01.408088 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47841e6a-8368-4d61-b20e-91888b74cd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:01.408372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47841e6a-8368-4d61-b20e-91888b74cd56 admin admin] PATH_INFO: `/v3/users/436b828f7aa04caba9083436538a8ffc` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:01.409573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47841e6a-8368-4d61-b20e-91888b74cd56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=436b828f7aa04caba9083436538a8ffc)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:17:01.411593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47841e6a-8368-4d61-b20e-91888b74cd56 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:17:01.569619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-47841e6a-8368-4d61-b20e-91888b74cd56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '436b828f7aa04caba9083436538a8ffc'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:01.570373 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-47841e6a-8368-4d61-b20e-91888b74cd56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '436b828f7aa04caba9083436538a8ffc'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:01.592558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-47841e6a-8368-4d61-b20e-91888b74cd56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '436b828f7aa04caba9083436538a8ffc'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:01.593319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-47841e6a-8368-4d61-b20e-91888b74cd56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '436b828f7aa04caba9083436538a8ffc'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:01.601895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-47841e6a-8368-4d61-b20e-91888b74cd56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '436b828f7aa04caba9083436538a8ffc'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:01.700439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1283/2531] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:17:01 2022] DELETE /identity/v3/users/436b828f7aa04caba9083436538a8ffc => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:17:01.704968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b540148-0a60-4e44-aeb5-c614bdad2d1b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:01.705447 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b540148-0a60-4e44-aeb5-c614bdad2d1b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:01.705538 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b540148-0a60-4e44-aeb5-c614bdad2d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:01.722914 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-1b540148-0a60-4e44-aeb5-c614bdad2d1b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:17:01.724636 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1b540148-0a60-4e44-aeb5-c614bdad2d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:17:01.782117 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1249/2532] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:17:01.792326 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47841e6a-8368-4d61-b20e-91888b74cd56 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:17:01.793805 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-47841e6a-8368-4d61-b20e-91888b74cd56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:17:01.854198 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47841e6a-8368-4d61-b20e-91888b74cd56 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:17:01.857983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938a54e-2256-4459-a585-544972febb95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:17:01.858641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938a54e-2256-4459-a585-544972febb95 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:01.858931 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938a54e-2256-4459-a585-544972febb95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:01.859260 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938a54e-2256-4459-a585-544972febb95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:01.869500 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938a54e-2256-4459-a585-544972febb95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:17:01.870949 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938a54e-2256-4459-a585-544972febb95 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:17:01.880826 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1284/2533] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:17:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:17:02.290073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b540148-0a60-4e44-aeb5-c614bdad2d1b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:17:02.291513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1b540148-0a60-4e44-aeb5-c614bdad2d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:17:02.304113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b540148-0a60-4e44-aeb5-c614bdad2d1b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:17:02.309775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd423c9b-b1ab-460e-98ce-59117df36a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:17:02.310980 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd423c9b-b1ab-460e-98ce-59117df36a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:02.311301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd423c9b-b1ab-460e-98ce-59117df36a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:02.311579 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd423c9b-b1ab-460e-98ce-59117df36a77 admin admin] PATH_INFO: `/v3/projects/b21daf298af44b17bd0b2b0fc92d10f5` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:02.312863 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd423c9b-b1ab-460e-98ce-59117df36a77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b21daf298af44b17bd0b2b0fc92d10f5)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:17:02.316103 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd423c9b-b1ab-460e-98ce-59117df36a77 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:17:02.408996 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-cd423c9b-b1ab-460e-98ce-59117df36a77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b21daf298af44b17bd0b2b0fc92d10f5', 'request_id': 'req-cd423c9b-b1ab-460e-98ce-59117df36a77'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:02.453519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-cd423c9b-b1ab-460e-98ce-59117df36a77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b21daf298af44b17bd0b2b0fc92d10f5', 'request_id': 'req-cd423c9b-b1ab-460e-98ce-59117df36a77'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:02.459824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-cd423c9b-b1ab-460e-98ce-59117df36a77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b21daf298af44b17bd0b2b0fc92d10f5', 'request_id': 'req-cd423c9b-b1ab-460e-98ce-59117df36a77'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:02.471129 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-cd423c9b-b1ab-460e-98ce-59117df36a77 admin admin] The token cache is being invalidate because project b21daf298af44b17bd0b2b0fc92d10f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:17:02.471470 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-cd423c9b-b1ab-460e-98ce-59117df36a77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:02.472190 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-cd423c9b-b1ab-460e-98ce-59117df36a77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:02.473725 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1250/2534] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:17:02 2022] DELETE /identity/v3/projects/b21daf298af44b17bd0b2b0fc92d10f5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:17:02.480090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec01e71-6f19-4a4f-9bcf-aa1fba7f051b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:02.480580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec01e71-6f19-4a4f-9bcf-aa1fba7f051b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:02.480686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec01e71-6f19-4a4f-9bcf-aa1fba7f051b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:02.498074 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5ec01e71-6f19-4a4f-9bcf-aa1fba7f051b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:17:02.499342 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5ec01e71-6f19-4a4f-9bcf-aa1fba7f051b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:17:02.516581 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:17:02.560221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1285/2535] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:17:02.569719 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1859ad-e25c-4c1b-82ab-e8ba97774c43 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:02.570013 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1859ad-e25c-4c1b-82ab-e8ba97774c43 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:02.570322 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1859ad-e25c-4c1b-82ab-e8ba97774c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:02.590187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-fc1859ad-e25c-4c1b-82ab-e8ba97774c43 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:17:02.591448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-fc1859ad-e25c-4c1b-82ab-e8ba97774c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:17:02.599689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1251/2536] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:17:02.604104 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec01e71-6f19-4a4f-9bcf-aa1fba7f051b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:17:02.613630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec01e71-6f19-4a4f-9bcf-aa1fba7f051b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:17:02.618234 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a0b1e-4135-4ef5-9dd7-f25ea46f6944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:17:02.618883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a0b1e-4135-4ef5-9dd7-f25ea46f6944 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:02.619200 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a0b1e-4135-4ef5-9dd7-f25ea46f6944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:02.619486 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a0b1e-4135-4ef5-9dd7-f25ea46f6944 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:02.620186 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a0b1e-4135-4ef5-9dd7-f25ea46f6944 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:17:02.620512 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a0b1e-4135-4ef5-9dd7-f25ea46f6944 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:17:02.622120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a0b1e-4135-4ef5-9dd7-f25ea46f6944 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:17:02.636049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1286/2537] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 19:17:02.640250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1859ad-e25c-4c1b-82ab-e8ba97774c43 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:17:02.650071 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1859ad-e25c-4c1b-82ab-e8ba97774c43 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:17:02.655069 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aedce9-e7e8-43f9-9028-4f2eb99b1679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:17:02.655743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aedce9-e7e8-43f9-9028-4f2eb99b1679 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:02.656026 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aedce9-e7e8-43f9-9028-4f2eb99b1679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:02.656298 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aedce9-e7e8-43f9-9028-4f2eb99b1679 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:02.656993 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aedce9-e7e8-43f9-9028-4f2eb99b1679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:17:02.658871 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aedce9-e7e8-43f9-9028-4f2eb99b1679 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:17:02.736335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1252/2538] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:17:02 2022] POST /identity/v3/projects => generated 367 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:17:02.740340 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a0b1e-4135-4ef5-9dd7-f25ea46f6944 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:17:02.751854 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a0b1e-4135-4ef5-9dd7-f25ea46f6944 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:17:02.811719 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b2e25-32b6-4808-9c89-f4d2f36be729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:17:02.812595 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b2e25-32b6-4808-9c89-f4d2f36be729 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:02.812696 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b2e25-32b6-4808-9c89-f4d2f36be729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:02.812945 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b2e25-32b6-4808-9c89-f4d2f36be729 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:02.813748 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b2e25-32b6-4808-9c89-f4d2f36be729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:17:02.815637 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b2e25-32b6-4808-9c89-f4d2f36be729 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:17:02.905967 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1287/2539] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:17:02 2022] POST /identity/v3/users => generated 284 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:17:02.911619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aedce9-e7e8-43f9-9028-4f2eb99b1679 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:17:02.922039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aedce9-e7e8-43f9-9028-4f2eb99b1679 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:17:02.927119 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4427c-2cc7-48cf-a595-b4dacf37536a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:17:02.927801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4427c-2cc7-48cf-a595-b4dacf37536a admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:02.928085 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4427c-2cc7-48cf-a595-b4dacf37536a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:02.928357 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4427c-2cc7-48cf-a595-b4dacf37536a admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:02.929080 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4427c-2cc7-48cf-a595-b4dacf37536a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:17:02.930660 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4427c-2cc7-48cf-a595-b4dacf37536a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:17:02.940967 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1253/2540] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:17:02.945436 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b2e25-32b6-4808-9c89-f4d2f36be729 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:17:02.955546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b2e25-32b6-4808-9c89-f4d2f36be729 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:17:02.960238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65596ea-03a0-4597-97d5-d266c2b810e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:17:02.960897 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65596ea-03a0-4597-97d5-d266c2b810e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:02.961185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65596ea-03a0-4597-97d5-d266c2b810e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:02.961467 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65596ea-03a0-4597-97d5-d266c2b810e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:02.962189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65596ea-03a0-4597-97d5-d266c2b810e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:17:02.964026 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65596ea-03a0-4597-97d5-d266c2b810e9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:17:02.974670 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1288/2541] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:17:02.978802 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4427c-2cc7-48cf-a595-b4dacf37536a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:17:02.989639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4427c-2cc7-48cf-a595-b4dacf37536a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:17:02.994253 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c17bd0-215f-4f58-8ffd-dbe4d1a743bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:17:02.995142 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c17bd0-215f-4f58-8ffd-dbe4d1a743bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:02.995463 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c17bd0-215f-4f58-8ffd-dbe4d1a743bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:02.995742 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c17bd0-215f-4f58-8ffd-dbe4d1a743bd admin admin] PATH_INFO: `/v3/projects/35f4e7026ef745a2af3a1a70390a0c89/users/f794fcc49c234c35aa745a6799bdec18/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:03.011108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c17bd0-215f-4f58-8ffd-dbe4d1a743bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=35f4e7026ef745a2af3a1a70390a0c89, user_id=f794fcc49c234c35aa745a6799bdec18, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:17:03.013784 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c17bd0-215f-4f58-8ffd-dbe4d1a743bd admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:17:03.082184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1254/2542] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:17:02 2022] PUT /identity/v3/projects/35f4e7026ef745a2af3a1a70390a0c89/users/f794fcc49c234c35aa745a6799bdec18/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:17:03.094132 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b6f2b-cc7e-47a9-b5b1-280d5ca6e173 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:03.094432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b6f2b-cc7e-47a9-b5b1-280d5ca6e173 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:03.094717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b6f2b-cc7e-47a9-b5b1-280d5ca6e173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:03.121987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-502b6f2b-cc7e-47a9-b5b1-280d5ca6e173 None None] MFA Rules not processed for user `f794fcc49c234c35aa745a6799bdec18`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:17:03.123218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-502b6f2b-cc7e-47a9-b5b1-280d5ca6e173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:17:03.185913 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1289/2543] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:17:03 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:17:03.197327 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c17bd0-215f-4f58-8ffd-dbe4d1a743bd admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:17:03.198987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-39c17bd0-215f-4f58-8ffd-dbe4d1a743bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:17:03.220322 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:17:03.256224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c17bd0-215f-4f58-8ffd-dbe4d1a743bd admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:17:03.259765 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f787e-549e-47d4-b408-e67de3805383 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5d76eae62b54521a95f36284118fada', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:17:03.263661 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f787e-549e-47d4-b408-e67de3805383 service glance] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:03.263661 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f787e-549e-47d4-b408-e67de3805383 service glance] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:03.263661 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f787e-549e-47d4-b408-e67de3805383 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:03.269136 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f787e-549e-47d4-b408-e67de3805383 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:17:03.271115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f787e-549e-47d4-b408-e67de3805383 service glance] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:17:03.280083 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1255/2544] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:17:03 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:17:13.183158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:17:13.309663 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=17 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:17:25.581607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de52b5-de80-4b06-927c-a3d49eb1a632 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:25.582129 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de52b5-de80-4b06-927c-a3d49eb1a632 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:25.582244 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de52b5-de80-4b06-927c-a3d49eb1a632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:25.600180 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e7de52b5-de80-4b06-927c-a3d49eb1a632 None None] MFA Rules not processed for user `89bf838da11b41bc9b9f77c9f10a6bbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:17:25.601412 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e7de52b5-de80-4b06-927c-a3d49eb1a632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:17:25.655783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1290/2545] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:17:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:17:35.648803 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=25 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:17:49.725188 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f787e-549e-47d4-b408-e67de3805383 service glance] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:17:49.726693 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-524f787e-549e-47d4-b408-e67de3805383 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:17:49.783567 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f787e-549e-47d4-b408-e67de3805383 service glance] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:17:49.787376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412c523-0176-420c-ae9a-8b36299fafda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:17:49.788029 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412c523-0176-420c-ae9a-8b36299fafda service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:49.788316 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412c523-0176-420c-ae9a-8b36299fafda service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:49.788594 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412c523-0176-420c-ae9a-8b36299fafda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:49.791216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3412c523-0176-420c-ae9a-8b36299fafda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:17:49.822365 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de52b5-de80-4b06-927c-a3d49eb1a632 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:17:49.823874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e7de52b5-de80-4b06-927c-a3d49eb1a632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:17:49.835606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de52b5-de80-4b06-927c-a3d49eb1a632 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:17:49.840454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c09a01-6cfa-4bcf-98f0-978459074c69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:17:49.841170 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c09a01-6cfa-4bcf-98f0-978459074c69 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:49.841451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c09a01-6cfa-4bcf-98f0-978459074c69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:49.841743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c09a01-6cfa-4bcf-98f0-978459074c69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:49.843890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c3c09a01-6cfa-4bcf-98f0-978459074c69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:17:49.882173 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412c523-0176-420c-ae9a-8b36299fafda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:17:49.883781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412c523-0176-420c-ae9a-8b36299fafda service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:17:49.906552 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1256/2546] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:17:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:17:49.923726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c09a01-6cfa-4bcf-98f0-978459074c69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:17:49.925192 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c09a01-6cfa-4bcf-98f0-978459074c69 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:17:49.938031 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1291/2547] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:17:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:17:50.918277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525da617-fbd6-4b81-ba9e-f7ee9a6d61e1 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:50.918772 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525da617-fbd6-4b81-ba9e-f7ee9a6d61e1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:50.918873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525da617-fbd6-4b81-ba9e-f7ee9a6d61e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:50.947994 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-525da617-fbd6-4b81-ba9e-f7ee9a6d61e1 None None] MFA Rules not processed for user `4ef0fad9a71d43bfbaba9c87dfed41c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:17:50.949236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-525da617-fbd6-4b81-ba9e-f7ee9a6d61e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:17:51.006972 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1257/2548] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 14 19:17:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:17:51.047097 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c09a01-6cfa-4bcf-98f0-978459074c69 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:17:51.048568 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c3c09a01-6cfa-4bcf-98f0-978459074c69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:17:51.123812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c09a01-6cfa-4bcf-98f0-978459074c69 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:17:51.129072 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332f64e-1b82-46b8-977a-a4eb80f47f04 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4368f68724f946eca53084a9ec7ff590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:17:51.129743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332f64e-1b82-46b8-977a-a4eb80f47f04 service swift] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:51.130029 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332f64e-1b82-46b8-977a-a4eb80f47f04 service swift] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:51.130312 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332f64e-1b82-46b8-977a-a4eb80f47f04 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:51.139894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332f64e-1b82-46b8-977a-a4eb80f47f04 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:17:51.141506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332f64e-1b82-46b8-977a-a4eb80f47f04 service swift] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:17:51.151181 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1292/2549] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 14 19:17:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 19:17:55.447475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525da617-fbd6-4b81-ba9e-f7ee9a6d61e1 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:17:55.460559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:17:55.521310 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525da617-fbd6-4b81-ba9e-f7ee9a6d61e1 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:17:55.526854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58800b-3001-4f00-99f3-1566c6d6d1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:17:55.527849 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58800b-3001-4f00-99f3-1566c6d6d1b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:55.528136 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58800b-3001-4f00-99f3-1566c6d6d1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:55.528409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58800b-3001-4f00-99f3-1566c6d6d1b6 admin admin] PATH_INFO: `/v3/users/f794fcc49c234c35aa745a6799bdec18` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:55.529610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58800b-3001-4f00-99f3-1566c6d6d1b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f794fcc49c234c35aa745a6799bdec18)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:17:55.531621 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58800b-3001-4f00-99f3-1566c6d6d1b6 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:17:55.697854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-fc58800b-3001-4f00-99f3-1566c6d6d1b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f794fcc49c234c35aa745a6799bdec18'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:55.721638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-fc58800b-3001-4f00-99f3-1566c6d6d1b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f794fcc49c234c35aa745a6799bdec18'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:55.722602 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-fc58800b-3001-4f00-99f3-1566c6d6d1b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f794fcc49c234c35aa745a6799bdec18'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:55.726511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-fc58800b-3001-4f00-99f3-1566c6d6d1b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f794fcc49c234c35aa745a6799bdec18'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:55.744769 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-fc58800b-3001-4f00-99f3-1566c6d6d1b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f794fcc49c234c35aa745a6799bdec18'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:55.814080 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1258/2550] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:17:55 2022] DELETE /identity/v3/users/f794fcc49c234c35aa745a6799bdec18 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:17:55.819265 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e97a3c-c39f-497e-a4c7-637164dbde3b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:55.820152 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e97a3c-c39f-497e-a4c7-637164dbde3b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:55.820447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e97a3c-c39f-497e-a4c7-637164dbde3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:55.839861 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-27e97a3c-c39f-497e-a4c7-637164dbde3b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:17:55.841667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-27e97a3c-c39f-497e-a4c7-637164dbde3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:17:55.916434 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1293/2551] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:17:55.928070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58800b-3001-4f00-99f3-1566c6d6d1b6 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:17:55.933384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-fc58800b-3001-4f00-99f3-1566c6d6d1b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:17:56.019822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58800b-3001-4f00-99f3-1566c6d6d1b6 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:17:56.024472 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6976f5c-55a2-4114-8744-329d15c10aca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:17:56.025254 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6976f5c-55a2-4114-8744-329d15c10aca service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:56.025773 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6976f5c-55a2-4114-8744-329d15c10aca service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:56.026156 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6976f5c-55a2-4114-8744-329d15c10aca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:56.037024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6976f5c-55a2-4114-8744-329d15c10aca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:17:56.038615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6976f5c-55a2-4114-8744-329d15c10aca service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:17:56.058010 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1259/2552] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:17:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:17:56.637651 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e97a3c-c39f-497e-a4c7-637164dbde3b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:17:56.639288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-27e97a3c-c39f-497e-a4c7-637164dbde3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:17:56.661527 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e97a3c-c39f-497e-a4c7-637164dbde3b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:17:56.669182 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480707ac-6615-4538-80d7-b253239d15b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:17:56.671017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480707ac-6615-4538-80d7-b253239d15b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:17:56.671017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480707ac-6615-4538-80d7-b253239d15b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:17:56.671017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480707ac-6615-4538-80d7-b253239d15b8 admin admin] PATH_INFO: `/v3/projects/35f4e7026ef745a2af3a1a70390a0c89` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:17:56.672473 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480707ac-6615-4538-80d7-b253239d15b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35f4e7026ef745a2af3a1a70390a0c89)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:17:56.673835 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480707ac-6615-4538-80d7-b253239d15b8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:17:56.686845 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:17:56.809983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-480707ac-6615-4538-80d7-b253239d15b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35f4e7026ef745a2af3a1a70390a0c89', 'request_id': 'req-480707ac-6615-4538-80d7-b253239d15b8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:56.881295 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-480707ac-6615-4538-80d7-b253239d15b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35f4e7026ef745a2af3a1a70390a0c89', 'request_id': 'req-480707ac-6615-4538-80d7-b253239d15b8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:56.888800 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-480707ac-6615-4538-80d7-b253239d15b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35f4e7026ef745a2af3a1a70390a0c89', 'request_id': 'req-480707ac-6615-4538-80d7-b253239d15b8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:56.902700 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-480707ac-6615-4538-80d7-b253239d15b8 admin admin] The token cache is being invalidate because project 35f4e7026ef745a2af3a1a70390a0c89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:17:56.907431 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-480707ac-6615-4538-80d7-b253239d15b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:56.907431 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-480707ac-6615-4538-80d7-b253239d15b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:17:56.907431 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1294/2553] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:17:56 2022] DELETE /identity/v3/projects/35f4e7026ef745a2af3a1a70390a0c89 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:18:06.046938 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=74,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:18:06.897502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:18:09.105910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2c5ce-987d-4a18-bf24-6912f9ab8126 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:09.106474 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2c5ce-987d-4a18-bf24-6912f9ab8126 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:09.106474 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2c5ce-987d-4a18-bf24-6912f9ab8126 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:09.107867 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1260/2554] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 14 19:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:18:09.114936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c738a9-5fe2-4154-bc0e-54a31d77bc18 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:09.115448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c738a9-5fe2-4154-bc0e-54a31d77bc18 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:09.115546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c738a9-5fe2-4154-bc0e-54a31d77bc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:09.152977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-40c738a9-5fe2-4154-bc0e-54a31d77bc18 None None] MFA Rules not processed for user `8d27d6df5cb64697adebd961dfbec324`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:18:09.154179 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-40c738a9-5fe2-4154-bc0e-54a31d77bc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:18:09.218914 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1295/2555] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 14 19:18:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:18:09.230185 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2c5ce-987d-4a18-bf24-6912f9ab8126 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:18:09.231891 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3cc2c5ce-987d-4a18-bf24-6912f9ab8126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:18:09.288972 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2c5ce-987d-4a18-bf24-6912f9ab8126 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:18:09.292539 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c590b9e-0936-4900-825f-7eec84e30026 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5d76eae62b54521a95f36284118fada', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:18:09.293180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c590b9e-0936-4900-825f-7eec84e30026 service glance] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:09.293466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c590b9e-0936-4900-825f-7eec84e30026 service glance] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:09.293744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c590b9e-0936-4900-825f-7eec84e30026 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:09.303590 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c590b9e-0936-4900-825f-7eec84e30026 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:18:09.305016 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c590b9e-0936-4900-825f-7eec84e30026 service glance] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:18:09.315946 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1261/2556] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:18:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:18:09.534880 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c58f6-01cd-4393-ab60-9dd634caaebd None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:09.535225 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c58f6-01cd-4393-ab60-9dd634caaebd None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:09.535511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c58f6-01cd-4393-ab60-9dd634caaebd None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:09.536901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1296/2557] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 14 19:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 19:18:09.542362 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0713632-4d09-473b-bd57-15389d24f299 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:09.542653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0713632-4d09-473b-bd57-15389d24f299 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:09.542929 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0713632-4d09-473b-bd57-15389d24f299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:09.578323 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c0713632-4d09-473b-bd57-15389d24f299 None None] MFA Rules not processed for user `74b8c75b9cce442b8c430fab6a3fea17`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:18:09.579548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c0713632-4d09-473b-bd57-15389d24f299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:18:09.629294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1262/2558] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 14 19:18:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:18:09.641784 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c58f6-01cd-4393-ab60-9dd634caaebd None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:18:09.645707 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-af4c58f6-01cd-4393-ab60-9dd634caaebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:18:09.657546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c58f6-01cd-4393-ab60-9dd634caaebd None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:18:09.661116 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71287f94-5736-48da-9f2e-a9e5d9eff912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:18:09.661745 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71287f94-5736-48da-9f2e-a9e5d9eff912 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:09.662029 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71287f94-5736-48da-9f2e-a9e5d9eff912 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:09.662307 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71287f94-5736-48da-9f2e-a9e5d9eff912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:09.669290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71287f94-5736-48da-9f2e-a9e5d9eff912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:18:09.670708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71287f94-5736-48da-9f2e-a9e5d9eff912 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:18:09.678282 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1297/2559] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:18:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:18:09.758531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877be2a-0acd-468f-aaab-53c67ee309cd None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:09.758837 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877be2a-0acd-468f-aaab-53c67ee309cd None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:09.759656 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877be2a-0acd-468f-aaab-53c67ee309cd None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:09.761066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1263/2560] 10.222.0.69 () {54 vars in 921 bytes} [Thu Jul 14 19:18:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 19:18:09.770146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe3efa-9cf7-469c-a4ab-786d0f3a3f24 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:09.770440 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe3efa-9cf7-469c-a4ab-786d0f3a3f24 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:09.770722 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe3efa-9cf7-469c-a4ab-786d0f3a3f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:09.790780 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-fcbe3efa-9cf7-469c-a4ab-786d0f3a3f24 None None] MFA Rules not processed for user `8d27d6df5cb64697adebd961dfbec324`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:18:09.792005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-fcbe3efa-9cf7-469c-a4ab-786d0f3a3f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:18:09.801115 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1298/2561] 10.222.0.69 () {58 vars in 1018 bytes} [Thu Jul 14 19:18:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:18:09.811462 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877be2a-0acd-468f-aaab-53c67ee309cd None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:18:09.822848 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877be2a-0acd-468f-aaab-53c67ee309cd None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:18:09.826514 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9099c7-964a-40e7-b434-a53e0ec0263a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:18:09.827152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9099c7-964a-40e7-b434-a53e0ec0263a service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:09.827469 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9099c7-964a-40e7-b434-a53e0ec0263a service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:09.827752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9099c7-964a-40e7-b434-a53e0ec0263a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:09.838408 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9099c7-964a-40e7-b434-a53e0ec0263a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:18:09.839871 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9099c7-964a-40e7-b434-a53e0ec0263a service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:18:09.851118 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1264/2562] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:18:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:18:12.270567 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d28df-9cfe-4426-9e71-f88da599cb01 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:12.270974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d28df-9cfe-4426-9e71-f88da599cb01 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:12.271126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d28df-9cfe-4426-9e71-f88da599cb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:12.322120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-104d28df-9cfe-4426-9e71-f88da599cb01 None None] MFA Rules not processed for user `4ef0fad9a71d43bfbaba9c87dfed41c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:18:12.323362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-104d28df-9cfe-4426-9e71-f88da599cb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:18:12.387031 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1299/2563] 10.222.0.69 () {58 vars in 1009 bytes} [Thu Jul 14 19:18:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:18:12.398972 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9099c7-964a-40e7-b434-a53e0ec0263a service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:18:12.400435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-da9099c7-964a-40e7-b434-a53e0ec0263a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:18:12.468242 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9099c7-964a-40e7-b434-a53e0ec0263a service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:18:12.473260 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81886a-a656-4032-afa6-125d00ea9ebe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4368f68724f946eca53084a9ec7ff590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:18:12.473906 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81886a-a656-4032-afa6-125d00ea9ebe service swift] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:12.474191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81886a-a656-4032-afa6-125d00ea9ebe service swift] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:12.474468 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81886a-a656-4032-afa6-125d00ea9ebe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:12.482909 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81886a-a656-4032-afa6-125d00ea9ebe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:18:12.484556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81886a-a656-4032-afa6-125d00ea9ebe service swift] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:18:12.493644 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1265/2564] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 14 19:18:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 19:18:22.383573 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=86 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:18:22.489870 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=93 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:18:35.454187 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d36dc-37f9-4591-9a54-17fd3863725c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:35.454648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d36dc-37f9-4591-9a54-17fd3863725c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:35.454812 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d36dc-37f9-4591-9a54-17fd3863725c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:35.472716 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-179d36dc-37f9-4591-9a54-17fd3863725c None None] MFA Rules not processed for user `4ef0fad9a71d43bfbaba9c87dfed41c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:18:35.474037 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-179d36dc-37f9-4591-9a54-17fd3863725c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:18:35.481388 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1300/2565] 10.222.0.69 () {58 vars in 967 bytes} [Thu Jul 14 19:18:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:18:35.494902 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81886a-a656-4032-afa6-125d00ea9ebe service swift] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:18:35.514883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81886a-a656-4032-afa6-125d00ea9ebe service swift] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:18:35.519808 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ff2a1-805c-4314-9849-b3644269ea5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4368f68724f946eca53084a9ec7ff590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:18:35.520437 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ff2a1-805c-4314-9849-b3644269ea5f service swift] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:35.520721 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ff2a1-805c-4314-9849-b3644269ea5f service swift] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:35.521004 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ff2a1-805c-4314-9849-b3644269ea5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:35.529345 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ff2a1-805c-4314-9849-b3644269ea5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:18:35.530968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ff2a1-805c-4314-9849-b3644269ea5f service swift] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:18:35.538980 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1266/2566] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 14 19:18:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 19:18:40.069165 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80380d6-352c-439e-8dea-9f8f6548d895 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:40.069600 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80380d6-352c-439e-8dea-9f8f6548d895 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:40.069729 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80380d6-352c-439e-8dea-9f8f6548d895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:40.087465 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d80380d6-352c-439e-8dea-9f8f6548d895 None None] MFA Rules not processed for user `4ef0fad9a71d43bfbaba9c87dfed41c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:18:40.088641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d80380d6-352c-439e-8dea-9f8f6548d895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:18:40.096093 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1301/2567] 10.222.0.69 () {58 vars in 967 bytes} [Thu Jul 14 19:18:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:18:40.108718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ff2a1-805c-4314-9849-b3644269ea5f service swift] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:18:40.119325 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ff2a1-805c-4314-9849-b3644269ea5f service swift] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:18:40.124381 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf466eb-801d-4fde-89b7-8a47df14d276 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4368f68724f946eca53084a9ec7ff590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:18:40.125017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf466eb-801d-4fde-89b7-8a47df14d276 service swift] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:40.125323 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf466eb-801d-4fde-89b7-8a47df14d276 service swift] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:40.125616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf466eb-801d-4fde-89b7-8a47df14d276 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:40.133622 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf466eb-801d-4fde-89b7-8a47df14d276 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:18:40.135252 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf466eb-801d-4fde-89b7-8a47df14d276 service swift] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:18:40.143099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1267/2568] 10.222.0.69 () {60 vars in 1384 bytes} [Thu Jul 14 19:18:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 19:18:48.414922 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80380d6-352c-439e-8dea-9f8f6548d895 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:18:48.416954 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d80380d6-352c-439e-8dea-9f8f6548d895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:18:48.489316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80380d6-352c-439e-8dea-9f8f6548d895 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:18:48.494191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fdcf66-9fdc-4e3a-a691-1627da6a0eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:18:48.495114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdcf66-9fdc-4e3a-a691-1627da6a0eff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:48.495430 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdcf66-9fdc-4e3a-a691-1627da6a0eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:48.495783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdcf66-9fdc-4e3a-a691-1627da6a0eff admin admin] PATH_INFO: `/v3/users/7f8022891f234799b8636ec47f7078c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:48.497029 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fdcf66-9fdc-4e3a-a691-1627da6a0eff admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f8022891f234799b8636ec47f7078c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:18:48.498975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fdcf66-9fdc-4e3a-a691-1627da6a0eff admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:18:48.690399 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-64fdcf66-9fdc-4e3a-a691-1627da6a0eff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f8022891f234799b8636ec47f7078c2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:48.691083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-64fdcf66-9fdc-4e3a-a691-1627da6a0eff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f8022891f234799b8636ec47f7078c2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:48.703564 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-64fdcf66-9fdc-4e3a-a691-1627da6a0eff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f8022891f234799b8636ec47f7078c2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:48.704240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-64fdcf66-9fdc-4e3a-a691-1627da6a0eff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f8022891f234799b8636ec47f7078c2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:48.711115 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-64fdcf66-9fdc-4e3a-a691-1627da6a0eff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f8022891f234799b8636ec47f7078c2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:48.778578 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1302/2569] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:18:48 2022] DELETE /identity/v3/users/7f8022891f234799b8636ec47f7078c2 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:18:49.035846 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf466eb-801d-4fde-89b7-8a47df14d276 service swift] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:18:49.046679 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf466eb-801d-4fde-89b7-8a47df14d276 service swift] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:18:49.106314 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b990cd5-76d4-403d-bd94-7b7df4e0a7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:18:49.107242 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b990cd5-76d4-403d-bd94-7b7df4e0a7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:49.107529 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b990cd5-76d4-403d-bd94-7b7df4e0a7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:49.107807 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b990cd5-76d4-403d-bd94-7b7df4e0a7c8 admin admin] PATH_INFO: `/v3/projects/c4b76c568f624909891661687bf457c4` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:49.109008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b990cd5-76d4-403d-bd94-7b7df4e0a7c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4b76c568f624909891661687bf457c4)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:18:49.110905 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b990cd5-76d4-403d-bd94-7b7df4e0a7c8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:18:49.246684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0b990cd5-76d4-403d-bd94-7b7df4e0a7c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4b76c568f624909891661687bf457c4', 'request_id': 'req-0b990cd5-76d4-403d-bd94-7b7df4e0a7c8'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:49.315280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0b990cd5-76d4-403d-bd94-7b7df4e0a7c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4b76c568f624909891661687bf457c4', 'request_id': 'req-0b990cd5-76d4-403d-bd94-7b7df4e0a7c8'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:49.321534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0b990cd5-76d4-403d-bd94-7b7df4e0a7c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4b76c568f624909891661687bf457c4', 'request_id': 'req-0b990cd5-76d4-403d-bd94-7b7df4e0a7c8'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:49.333387 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0b990cd5-76d4-403d-bd94-7b7df4e0a7c8 admin admin] The token cache is being invalidate because project c4b76c568f624909891661687bf457c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:18:49.333679 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0b990cd5-76d4-403d-bd94-7b7df4e0a7c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:49.334372 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0b990cd5-76d4-403d-bd94-7b7df4e0a7c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:49.335932 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1268/2570] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:18:49 2022] DELETE /identity/v3/projects/c4b76c568f624909891661687bf457c4 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:18:49.339935 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fdcf66-9fdc-4e3a-a691-1627da6a0eff admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:18:49.341351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-64fdcf66-9fdc-4e3a-a691-1627da6a0eff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:18:49.354923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fdcf66-9fdc-4e3a-a691-1627da6a0eff admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:18:49.409130 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de3589-223e-4117-9190-0766052b03dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:18:49.410041 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de3589-223e-4117-9190-0766052b03dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:49.410354 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de3589-223e-4117-9190-0766052b03dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:49.410631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de3589-223e-4117-9190-0766052b03dd admin admin] PATH_INFO: `/v3/users/c1fb89ab4d6d4433b9f8bbad00dc2e2f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:49.411890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de3589-223e-4117-9190-0766052b03dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1fb89ab4d6d4433b9f8bbad00dc2e2f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:18:49.413809 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de3589-223e-4117-9190-0766052b03dd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:18:49.419694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:18:49.568713 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a2de3589-223e-4117-9190-0766052b03dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1fb89ab4d6d4433b9f8bbad00dc2e2f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:49.569342 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a2de3589-223e-4117-9190-0766052b03dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1fb89ab4d6d4433b9f8bbad00dc2e2f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:49.581416 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a2de3589-223e-4117-9190-0766052b03dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1fb89ab4d6d4433b9f8bbad00dc2e2f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:49.582062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a2de3589-223e-4117-9190-0766052b03dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1fb89ab4d6d4433b9f8bbad00dc2e2f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:49.589376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a2de3589-223e-4117-9190-0766052b03dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1fb89ab4d6d4433b9f8bbad00dc2e2f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:49.670569 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1303/2571] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:18:49 2022] DELETE /identity/v3/users/c1fb89ab4d6d4433b9f8bbad00dc2e2f => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:18:49.963274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b990cd5-76d4-403d-bd94-7b7df4e0a7c8 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:18:49.964733 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0b990cd5-76d4-403d-bd94-7b7df4e0a7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:18:49.981199 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b990cd5-76d4-403d-bd94-7b7df4e0a7c8 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:18:50.034140 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:18:50.041710 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511335a-0068-4f62-9dce-2d65bf8abdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:18:50.042603 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511335a-0068-4f62-9dce-2d65bf8abdc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:50.042889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511335a-0068-4f62-9dce-2d65bf8abdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:50.043208 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511335a-0068-4f62-9dce-2d65bf8abdc6 admin admin] PATH_INFO: `/v3/projects/77ba62e66bda4f2aa7cc436186ffe794` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:50.044383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511335a-0068-4f62-9dce-2d65bf8abdc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77ba62e66bda4f2aa7cc436186ffe794)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:18:50.046493 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511335a-0068-4f62-9dce-2d65bf8abdc6 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:18:50.161681 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e511335a-0068-4f62-9dce-2d65bf8abdc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77ba62e66bda4f2aa7cc436186ffe794', 'request_id': 'req-e511335a-0068-4f62-9dce-2d65bf8abdc6'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:50.229711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e511335a-0068-4f62-9dce-2d65bf8abdc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77ba62e66bda4f2aa7cc436186ffe794', 'request_id': 'req-e511335a-0068-4f62-9dce-2d65bf8abdc6'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:50.235938 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e511335a-0068-4f62-9dce-2d65bf8abdc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77ba62e66bda4f2aa7cc436186ffe794', 'request_id': 'req-e511335a-0068-4f62-9dce-2d65bf8abdc6'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:50.247144 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e511335a-0068-4f62-9dce-2d65bf8abdc6 admin admin] The token cache is being invalidate because project 77ba62e66bda4f2aa7cc436186ffe794 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:18:50.247464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e511335a-0068-4f62-9dce-2d65bf8abdc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:50.248120 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e511335a-0068-4f62-9dce-2d65bf8abdc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:18:50.249688 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1269/2572] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:18:49 2022] DELETE /identity/v3/projects/77ba62e66bda4f2aa7cc436186ffe794 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:18:52.324517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de3589-223e-4117-9190-0766052b03dd admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:18:52.326139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a2de3589-223e-4117-9190-0766052b03dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:18:52.407443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de3589-223e-4117-9190-0766052b03dd admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:18:52.486049 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed84a6-26b6-4f45-a7d9-6405e472cd0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:18:52.486964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed84a6-26b6-4f45-a7d9-6405e472cd0f service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:52.487286 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed84a6-26b6-4f45-a7d9-6405e472cd0f service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:52.487594 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed84a6-26b6-4f45-a7d9-6405e472cd0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:52.489753 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-faed84a6-26b6-4f45-a7d9-6405e472cd0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:18:52.502924 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed84a6-26b6-4f45-a7d9-6405e472cd0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:18:52.505149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed84a6-26b6-4f45-a7d9-6405e472cd0f service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:18:52.518768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1304/2573] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:18:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:18:58.451968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511335a-0068-4f62-9dce-2d65bf8abdc6 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:18:58.453379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e511335a-0068-4f62-9dce-2d65bf8abdc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:18:58.508772 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511335a-0068-4f62-9dce-2d65bf8abdc6 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:18:58.512894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb422119-a77d-4d09-88f3-4b38c735c585 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:18:58.513563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb422119-a77d-4d09-88f3-4b38c735c585 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:18:58.513850 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb422119-a77d-4d09-88f3-4b38c735c585 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:18:58.514127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb422119-a77d-4d09-88f3-4b38c735c585 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:18:58.515875 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-bb422119-a77d-4d09-88f3-4b38c735c585 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:18:58.579015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb422119-a77d-4d09-88f3-4b38c735c585 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:18:58.580482 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb422119-a77d-4d09-88f3-4b38c735c585 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:18:58.590844 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1270/2574] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:18:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:19:01.816466 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed84a6-26b6-4f45-a7d9-6405e472cd0f service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:01.818157 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-faed84a6-26b6-4f45-a7d9-6405e472cd0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:01.830289 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed84a6-26b6-4f45-a7d9-6405e472cd0f service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:01.833966 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ec98c-4f13-43c9-8251-3e0fc43c0234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:01.834607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ec98c-4f13-43c9-8251-3e0fc43c0234 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:01.834893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ec98c-4f13-43c9-8251-3e0fc43c0234 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:01.835205 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ec98c-4f13-43c9-8251-3e0fc43c0234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:01.836978 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-091ec98c-4f13-43c9-8251-3e0fc43c0234 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:01.844628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:01.847371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ec98c-4f13-43c9-8251-3e0fc43c0234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:01.848941 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ec98c-4f13-43c9-8251-3e0fc43c0234 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:01.858108 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1305/2575] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:19:02.188800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb422119-a77d-4d09-88f3-4b38c735c585 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:02.190365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-bb422119-a77d-4d09-88f3-4b38c735c585 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:02.203871 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb422119-a77d-4d09-88f3-4b38c735c585 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:02.208635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a3204-01a9-4180-bd24-2aa542097b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:02.209597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a3204-01a9-4180-bd24-2aa542097b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:02.209880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a3204-01a9-4180-bd24-2aa542097b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:02.210176 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a3204-01a9-4180-bd24-2aa542097b15 admin admin] PATH_INFO: `/v3/users/7d5fa180ee554e8b8e46321892969489` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:02.225230 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a3204-01a9-4180-bd24-2aa542097b15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d5fa180ee554e8b8e46321892969489)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:02.227309 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a3204-01a9-4180-bd24-2aa542097b15 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:02.250595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:02.431526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b20a3204-01a9-4180-bd24-2aa542097b15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d5fa180ee554e8b8e46321892969489'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:02.439412 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b20a3204-01a9-4180-bd24-2aa542097b15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d5fa180ee554e8b8e46321892969489'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:02.440287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b20a3204-01a9-4180-bd24-2aa542097b15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d5fa180ee554e8b8e46321892969489'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:02.440932 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b20a3204-01a9-4180-bd24-2aa542097b15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d5fa180ee554e8b8e46321892969489'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:02.452631 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b20a3204-01a9-4180-bd24-2aa542097b15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d5fa180ee554e8b8e46321892969489'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:02.521115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1271/2576] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:19:02 2022] DELETE /identity/v3/users/7d5fa180ee554e8b8e46321892969489 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:02.768441 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ec98c-4f13-43c9-8251-3e0fc43c0234 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:02.781067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ec98c-4f13-43c9-8251-3e0fc43c0234 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:02.841476 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497811b1-1e7f-4451-843f-ba61409fd47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:02.842347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497811b1-1e7f-4451-843f-ba61409fd47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:02.842635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497811b1-1e7f-4451-843f-ba61409fd47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:02.842913 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497811b1-1e7f-4451-843f-ba61409fd47a admin admin] PATH_INFO: `/v3/projects/aa66aa59ee324394a45b3059ae1f49b5` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:02.853669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497811b1-1e7f-4451-843f-ba61409fd47a admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa66aa59ee324394a45b3059ae1f49b5)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:02.855677 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497811b1-1e7f-4451-843f-ba61409fd47a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:02.978242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-497811b1-1e7f-4451-843f-ba61409fd47a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa66aa59ee324394a45b3059ae1f49b5', 'request_id': 'req-497811b1-1e7f-4451-843f-ba61409fd47a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:03.035189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-497811b1-1e7f-4451-843f-ba61409fd47a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa66aa59ee324394a45b3059ae1f49b5', 'request_id': 'req-497811b1-1e7f-4451-843f-ba61409fd47a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:03.041191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-497811b1-1e7f-4451-843f-ba61409fd47a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa66aa59ee324394a45b3059ae1f49b5', 'request_id': 'req-497811b1-1e7f-4451-843f-ba61409fd47a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:03.052708 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-497811b1-1e7f-4451-843f-ba61409fd47a admin admin] The token cache is being invalidate because project aa66aa59ee324394a45b3059ae1f49b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:19:03.052990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-497811b1-1e7f-4451-843f-ba61409fd47a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:03.053621 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-497811b1-1e7f-4451-843f-ba61409fd47a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:03.055154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1306/2577] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:19:02 2022] DELETE /identity/v3/projects/aa66aa59ee324394a45b3059ae1f49b5 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:03.060512 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb0c02-7e9f-4856-8afb-1e429b147648 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:03.060816 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb0c02-7e9f-4856-8afb-1e429b147648 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:03.061144 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb0c02-7e9f-4856-8afb-1e429b147648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:03.079537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c7bb0c02-7e9f-4856-8afb-1e429b147648 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:03.080753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c7bb0c02-7e9f-4856-8afb-1e429b147648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:03.138773 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1272/2578] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:03.146645 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd32eea-419d-48aa-83fb-6e9e12588f7d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:03.146942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd32eea-419d-48aa-83fb-6e9e12588f7d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:03.147255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd32eea-419d-48aa-83fb-6e9e12588f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:03.165653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7dd32eea-419d-48aa-83fb-6e9e12588f7d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:03.166904 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7dd32eea-419d-48aa-83fb-6e9e12588f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:03.174803 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1307/2579] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:03.178786 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb0c02-7e9f-4856-8afb-1e429b147648 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:03.188809 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb0c02-7e9f-4856-8afb-1e429b147648 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:03.193392 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb924468-82ac-42db-8b07-e6068066462b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:03.194043 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb924468-82ac-42db-8b07-e6068066462b admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:03.194358 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb924468-82ac-42db-8b07-e6068066462b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:03.194638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb924468-82ac-42db-8b07-e6068066462b admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:03.195394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb924468-82ac-42db-8b07-e6068066462b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:19:03.195705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb924468-82ac-42db-8b07-e6068066462b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:03.197360 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb924468-82ac-42db-8b07-e6068066462b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:03.209332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1273/2580] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:19:03.213370 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd32eea-419d-48aa-83fb-6e9e12588f7d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:03.222436 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd32eea-419d-48aa-83fb-6e9e12588f7d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:03.227067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7de496-bcf7-4b96-870d-80c5fb829239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:03.227742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7de496-bcf7-4b96-870d-80c5fb829239 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:03.228035 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7de496-bcf7-4b96-870d-80c5fb829239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:03.228381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7de496-bcf7-4b96-870d-80c5fb829239 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:03.229057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7de496-bcf7-4b96-870d-80c5fb829239 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:03.230961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7de496-bcf7-4b96-870d-80c5fb829239 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:03.316053 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1308/2581] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:19:03 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:19:03.320167 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb924468-82ac-42db-8b07-e6068066462b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:03.330668 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb924468-82ac-42db-8b07-e6068066462b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:03.384785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bb1a4-3eda-4411-bd17-8d44864b187b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:03.386016 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bb1a4-3eda-4411-bd17-8d44864b187b admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:03.386304 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bb1a4-3eda-4411-bd17-8d44864b187b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:03.386581 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bb1a4-3eda-4411-bd17-8d44864b187b admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:03.387299 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54bb1a4-3eda-4411-bd17-8d44864b187b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:03.389286 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54bb1a4-3eda-4411-bd17-8d44864b187b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:03.480945 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1274/2582] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:19:03 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:19:03.484996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7de496-bcf7-4b96-870d-80c5fb829239 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:03.495980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7de496-bcf7-4b96-870d-80c5fb829239 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:03.500623 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb436190-f560-4362-8bfe-1db80d29497a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:03.501286 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb436190-f560-4362-8bfe-1db80d29497a admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:03.501576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb436190-f560-4362-8bfe-1db80d29497a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:03.501859 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb436190-f560-4362-8bfe-1db80d29497a admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:03.502662 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb436190-f560-4362-8bfe-1db80d29497a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:03.504311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb436190-f560-4362-8bfe-1db80d29497a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:03.514893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1309/2583] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:03.518767 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bb1a4-3eda-4411-bd17-8d44864b187b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:03.528989 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bb1a4-3eda-4411-bd17-8d44864b187b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:03.533643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fac9a2-2f93-46b3-814a-84f2f12d3778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:03.534301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fac9a2-2f93-46b3-814a-84f2f12d3778 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:03.534593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fac9a2-2f93-46b3-814a-84f2f12d3778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:03.534920 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fac9a2-2f93-46b3-814a-84f2f12d3778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:03.535687 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fac9a2-2f93-46b3-814a-84f2f12d3778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:03.537294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fac9a2-2f93-46b3-814a-84f2f12d3778 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:03.547967 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1275/2584] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:03.554908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb436190-f560-4362-8bfe-1db80d29497a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:03.562495 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb436190-f560-4362-8bfe-1db80d29497a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:03.567112 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80007c2c-e8a5-47b0-918e-9b5a8adfda17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:03.568009 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80007c2c-e8a5-47b0-918e-9b5a8adfda17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:03.568299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80007c2c-e8a5-47b0-918e-9b5a8adfda17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:03.568580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80007c2c-e8a5-47b0-918e-9b5a8adfda17 admin admin] PATH_INFO: `/v3/projects/4059f88fc5924c2eb63deee1d1afdb0d/users/d4b4de47d3f942afb4401847e8102112/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:03.583002 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80007c2c-e8a5-47b0-918e-9b5a8adfda17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4059f88fc5924c2eb63deee1d1afdb0d, user_id=d4b4de47d3f942afb4401847e8102112, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:03.585688 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80007c2c-e8a5-47b0-918e-9b5a8adfda17 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:03.680541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1310/2585] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:19:03 2022] PUT /identity/v3/projects/4059f88fc5924c2eb63deee1d1afdb0d/users/d4b4de47d3f942afb4401847e8102112/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:03.694555 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296413b1-16ce-44c2-a991-ad1a5000b593 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:03.695096 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296413b1-16ce-44c2-a991-ad1a5000b593 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:03.695652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296413b1-16ce-44c2-a991-ad1a5000b593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:03.703962 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:03.726235 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-296413b1-16ce-44c2-a991-ad1a5000b593 None None] MFA Rules not processed for user `d4b4de47d3f942afb4401847e8102112`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:03.727470 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-296413b1-16ce-44c2-a991-ad1a5000b593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:03.789153 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1276/2586] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:03.798301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80007c2c-e8a5-47b0-918e-9b5a8adfda17 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:03.803934 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:03.858995 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80007c2c-e8a5-47b0-918e-9b5a8adfda17 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:03.863821 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20f296-cb42-45f4-99f2-9776d0de75fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:03.864482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20f296-cb42-45f4-99f2-9776d0de75fd admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:03.864776 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20f296-cb42-45f4-99f2-9776d0de75fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:03.865057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20f296-cb42-45f4-99f2-9776d0de75fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:03.865765 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20f296-cb42-45f4-99f2-9776d0de75fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:03.867748 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20f296-cb42-45f4-99f2-9776d0de75fd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:03.996854 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1311/2587] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:19:03 2022] POST /identity/v3/projects => generated 381 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:19:04.000922 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296413b1-16ce-44c2-a991-ad1a5000b593 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:04.011684 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296413b1-16ce-44c2-a991-ad1a5000b593 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:04.070603 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18740e86-fa48-440a-8120-f3e9d2742d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:04.071195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18740e86-fa48-440a-8120-f3e9d2742d54 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:04.071492 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18740e86-fa48-440a-8120-f3e9d2742d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:04.071770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18740e86-fa48-440a-8120-f3e9d2742d54 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:04.072536 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18740e86-fa48-440a-8120-f3e9d2742d54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:04.074598 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18740e86-fa48-440a-8120-f3e9d2742d54 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:04.155927 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1277/2588] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:19:03 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:19:04.160007 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20f296-cb42-45f4-99f2-9776d0de75fd admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:04.170779 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20f296-cb42-45f4-99f2-9776d0de75fd admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:04.175886 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d041e3-79bf-4876-9438-d777e303256c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:04.176062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d041e3-79bf-4876-9438-d777e303256c admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:04.176260 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d041e3-79bf-4876-9438-d777e303256c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:04.176538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d041e3-79bf-4876-9438-d777e303256c admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:04.177256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d041e3-79bf-4876-9438-d777e303256c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:04.178842 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d041e3-79bf-4876-9438-d777e303256c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:04.189328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1312/2589] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:04.193361 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18740e86-fa48-440a-8120-f3e9d2742d54 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:04.203056 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18740e86-fa48-440a-8120-f3e9d2742d54 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:04.207681 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d471d-fb13-4cbd-b175-10e09bd8fb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:04.208550 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d471d-fb13-4cbd-b175-10e09bd8fb27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:04.208838 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d471d-fb13-4cbd-b175-10e09bd8fb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:04.209118 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d471d-fb13-4cbd-b175-10e09bd8fb27 admin admin] PATH_INFO: `/v3/projects/ac9ed3ea15414cd89d9a011496aa26ab/users/87f273bf368e417e92a7247a8b1de5a4/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:04.223989 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d471d-fb13-4cbd-b175-10e09bd8fb27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac9ed3ea15414cd89d9a011496aa26ab, user_id=87f273bf368e417e92a7247a8b1de5a4, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:04.226632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d471d-fb13-4cbd-b175-10e09bd8fb27 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:04.298395 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1278/2590] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:19:04 2022] PUT /identity/v3/projects/ac9ed3ea15414cd89d9a011496aa26ab/users/87f273bf368e417e92a7247a8b1de5a4/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:04.302345 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d041e3-79bf-4876-9438-d777e303256c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:04.312222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d041e3-79bf-4876-9438-d777e303256c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:04.368009 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a68e78-1ecb-497e-8d25-3d48c7b290da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:04.368740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a68e78-1ecb-497e-8d25-3d48c7b290da admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:04.369027 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a68e78-1ecb-497e-8d25-3d48c7b290da admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:04.369302 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a68e78-1ecb-497e-8d25-3d48c7b290da admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:04.370021 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a68e78-1ecb-497e-8d25-3d48c7b290da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:04.371650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a68e78-1ecb-497e-8d25-3d48c7b290da admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:04.381538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1313/2591] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:04.386251 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d471d-fb13-4cbd-b175-10e09bd8fb27 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:04.398205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d471d-fb13-4cbd-b175-10e09bd8fb27 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:04.404076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24265e7-de01-46b9-94c8-a8330d6d9a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:04.404948 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24265e7-de01-46b9-94c8-a8330d6d9a63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:04.405233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24265e7-de01-46b9-94c8-a8330d6d9a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:04.405546 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24265e7-de01-46b9-94c8-a8330d6d9a63 admin admin] PATH_INFO: `/v3/projects/ac9ed3ea15414cd89d9a011496aa26ab/users/87f273bf368e417e92a7247a8b1de5a4/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:04.407545 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24265e7-de01-46b9-94c8-a8330d6d9a63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac9ed3ea15414cd89d9a011496aa26ab, user_id=87f273bf368e417e92a7247a8b1de5a4, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:04.410191 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24265e7-de01-46b9-94c8-a8330d6d9a63 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:04.420374 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1279/2592] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:19:04 2022] PUT /identity/v3/projects/ac9ed3ea15414cd89d9a011496aa26ab/users/87f273bf368e417e92a7247a8b1de5a4/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:04.431568 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587d78a-2cbd-4b8b-b5e6-cfc78e488a40 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:04.431868 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587d78a-2cbd-4b8b-b5e6-cfc78e488a40 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:04.432202 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587d78a-2cbd-4b8b-b5e6-cfc78e488a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:04.458363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c587d78a-2cbd-4b8b-b5e6-cfc78e488a40 None None] MFA Rules not processed for user `87f273bf368e417e92a7247a8b1de5a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:04.459604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c587d78a-2cbd-4b8b-b5e6-cfc78e488a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:04.527606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:04.531232 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1314/2593] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:04.542570 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24265e7-de01-46b9-94c8-a8330d6d9a63 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:04.544306 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f24265e7-de01-46b9-94c8-a8330d6d9a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:04.591002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:04.601675 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24265e7-de01-46b9-94c8-a8330d6d9a63 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:04.605375 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3663bf-2a7f-43b5-810d-8d900fa3fa21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:04.606076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3663bf-2a7f-43b5-810d-8d900fa3fa21 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:04.606365 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3663bf-2a7f-43b5-810d-8d900fa3fa21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:04.606645 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3663bf-2a7f-43b5-810d-8d900fa3fa21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:04.615642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3663bf-2a7f-43b5-810d-8d900fa3fa21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:04.617067 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3663bf-2a7f-43b5-810d-8d900fa3fa21 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:04.628350 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1280/2594] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:19:04.812409 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587d78a-2cbd-4b8b-b5e6-cfc78e488a40 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:04.821992 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587d78a-2cbd-4b8b-b5e6-cfc78e488a40 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:04.826554 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce78c8-9910-4ade-8f14-5bfd2b71398d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:04.827254 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce78c8-9910-4ade-8f14-5bfd2b71398d service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:04.827545 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce78c8-9910-4ade-8f14-5bfd2b71398d service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:04.827822 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce78c8-9910-4ade-8f14-5bfd2b71398d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:04.883206 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce78c8-9910-4ade-8f14-5bfd2b71398d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:04.884599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce78c8-9910-4ade-8f14-5bfd2b71398d service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:04.894114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1315/2595] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:19:14.707763 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=6 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:14.890426 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=28 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:17.416423 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3663bf-2a7f-43b5-810d-8d900fa3fa21 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:17.476689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3663bf-2a7f-43b5-810d-8d900fa3fa21 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:17.481210 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98353d3a-4a90-45de-9181-4065faf59eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:17.482180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98353d3a-4a90-45de-9181-4065faf59eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:17.482469 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98353d3a-4a90-45de-9181-4065faf59eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:17.482746 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98353d3a-4a90-45de-9181-4065faf59eda admin admin] PATH_INFO: `/v3/users/d4b4de47d3f942afb4401847e8102112` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:17.483957 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98353d3a-4a90-45de-9181-4065faf59eda admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4b4de47d3f942afb4401847e8102112)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:17.485947 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98353d3a-4a90-45de-9181-4065faf59eda admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:17.622925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-98353d3a-4a90-45de-9181-4065faf59eda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4b4de47d3f942afb4401847e8102112'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:17.629801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-98353d3a-4a90-45de-9181-4065faf59eda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4b4de47d3f942afb4401847e8102112'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:17.630466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-98353d3a-4a90-45de-9181-4065faf59eda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4b4de47d3f942afb4401847e8102112'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:17.631213 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-98353d3a-4a90-45de-9181-4065faf59eda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4b4de47d3f942afb4401847e8102112'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:17.642813 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-98353d3a-4a90-45de-9181-4065faf59eda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4b4de47d3f942afb4401847e8102112'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:17.732183 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1281/2596] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:19:17 2022] DELETE /identity/v3/users/d4b4de47d3f942afb4401847e8102112 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:17.736781 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c715c5a-146b-4b55-81f0-547c4795ab54 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:17.737263 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c715c5a-146b-4b55-81f0-547c4795ab54 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:17.737363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c715c5a-146b-4b55-81f0-547c4795ab54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:17.754235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-8c715c5a-146b-4b55-81f0-547c4795ab54 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:17.755492 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8c715c5a-146b-4b55-81f0-547c4795ab54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:17.815715 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1316/2597] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:17.825953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98353d3a-4a90-45de-9181-4065faf59eda admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:17.827002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-98353d3a-4a90-45de-9181-4065faf59eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:17.890277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98353d3a-4a90-45de-9181-4065faf59eda admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:17.894356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc6586-7462-4599-a507-b5c482615d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:17.895018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc6586-7462-4599-a507-b5c482615d22 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:17.895342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc6586-7462-4599-a507-b5c482615d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:17.895622 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc6586-7462-4599-a507-b5c482615d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:17.906737 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc6586-7462-4599-a507-b5c482615d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:17.908244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc6586-7462-4599-a507-b5c482615d22 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:17.919473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1282/2598] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:19:18.384699 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c715c5a-146b-4b55-81f0-547c4795ab54 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:18.386799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8c715c5a-146b-4b55-81f0-547c4795ab54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:18.399826 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c715c5a-146b-4b55-81f0-547c4795ab54 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:18.404951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df69e88-f687-4290-bd7a-5516fb9d9b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:18.405819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df69e88-f687-4290-bd7a-5516fb9d9b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:18.406144 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df69e88-f687-4290-bd7a-5516fb9d9b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:18.406431 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df69e88-f687-4290-bd7a-5516fb9d9b50 admin admin] PATH_INFO: `/v3/projects/4059f88fc5924c2eb63deee1d1afdb0d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:18.407726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df69e88-f687-4290-bd7a-5516fb9d9b50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4059f88fc5924c2eb63deee1d1afdb0d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:18.409705 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df69e88-f687-4290-bd7a-5516fb9d9b50 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:18.508412 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7df69e88-f687-4290-bd7a-5516fb9d9b50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4059f88fc5924c2eb63deee1d1afdb0d', 'request_id': 'req-7df69e88-f687-4290-bd7a-5516fb9d9b50'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:18.565955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7df69e88-f687-4290-bd7a-5516fb9d9b50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4059f88fc5924c2eb63deee1d1afdb0d', 'request_id': 'req-7df69e88-f687-4290-bd7a-5516fb9d9b50'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:18.571462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7df69e88-f687-4290-bd7a-5516fb9d9b50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4059f88fc5924c2eb63deee1d1afdb0d', 'request_id': 'req-7df69e88-f687-4290-bd7a-5516fb9d9b50'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:18.582893 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7df69e88-f687-4290-bd7a-5516fb9d9b50 admin admin] The token cache is being invalidate because project 4059f88fc5924c2eb63deee1d1afdb0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:19:18.583181 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7df69e88-f687-4290-bd7a-5516fb9d9b50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:18.583805 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7df69e88-f687-4290-bd7a-5516fb9d9b50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:18.585316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1317/2599] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:19:18 2022] DELETE /identity/v3/projects/4059f88fc5924c2eb63deee1d1afdb0d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:18.590017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc6586-7462-4599-a507-b5c482615d22 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:18.599685 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc6586-7462-4599-a507-b5c482615d22 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:18.617589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:18.655748 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead8937-ad7b-4de6-a42f-4ac1a0b89674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:18.656609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead8937-ad7b-4de6-a42f-4ac1a0b89674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:18.656894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead8937-ad7b-4de6-a42f-4ac1a0b89674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:18.657178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead8937-ad7b-4de6-a42f-4ac1a0b89674 admin admin] PATH_INFO: `/v3/users/87f273bf368e417e92a7247a8b1de5a4` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:18.658355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead8937-ad7b-4de6-a42f-4ac1a0b89674 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87f273bf368e417e92a7247a8b1de5a4)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:18.660312 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead8937-ad7b-4de6-a42f-4ac1a0b89674 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:18.860106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-eead8937-ad7b-4de6-a42f-4ac1a0b89674 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87f273bf368e417e92a7247a8b1de5a4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:18.867059 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-eead8937-ad7b-4de6-a42f-4ac1a0b89674 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87f273bf368e417e92a7247a8b1de5a4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:18.867770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-eead8937-ad7b-4de6-a42f-4ac1a0b89674 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87f273bf368e417e92a7247a8b1de5a4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:18.868394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-eead8937-ad7b-4de6-a42f-4ac1a0b89674 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87f273bf368e417e92a7247a8b1de5a4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:18.880838 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-eead8937-ad7b-4de6-a42f-4ac1a0b89674 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87f273bf368e417e92a7247a8b1de5a4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:18.966716 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1283/2600] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:19:18 2022] DELETE /identity/v3/users/87f273bf368e417e92a7247a8b1de5a4 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:19.379311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df69e88-f687-4290-bd7a-5516fb9d9b50 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:19.380864 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7df69e88-f687-4290-bd7a-5516fb9d9b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:19.395065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df69e88-f687-4290-bd7a-5516fb9d9b50 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:19.452920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0446c7e-bf01-41b4-8e78-956cca438bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:19.453787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0446c7e-bf01-41b4-8e78-956cca438bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:19.454072 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0446c7e-bf01-41b4-8e78-956cca438bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:19.454346 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0446c7e-bf01-41b4-8e78-956cca438bed admin admin] PATH_INFO: `/v3/projects/ac9ed3ea15414cd89d9a011496aa26ab` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:19.455580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0446c7e-bf01-41b4-8e78-956cca438bed admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac9ed3ea15414cd89d9a011496aa26ab)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:19.457515 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0446c7e-bf01-41b4-8e78-956cca438bed admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:19.522201 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:19.539320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f0446c7e-bf01-41b4-8e78-956cca438bed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac9ed3ea15414cd89d9a011496aa26ab', 'request_id': 'req-f0446c7e-bf01-41b4-8e78-956cca438bed'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:19.611599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f0446c7e-bf01-41b4-8e78-956cca438bed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac9ed3ea15414cd89d9a011496aa26ab', 'request_id': 'req-f0446c7e-bf01-41b4-8e78-956cca438bed'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:19.617289 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f0446c7e-bf01-41b4-8e78-956cca438bed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac9ed3ea15414cd89d9a011496aa26ab', 'request_id': 'req-f0446c7e-bf01-41b4-8e78-956cca438bed'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:19.628637 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f0446c7e-bf01-41b4-8e78-956cca438bed admin admin] The token cache is being invalidate because project ac9ed3ea15414cd89d9a011496aa26ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:19:19.628925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f0446c7e-bf01-41b4-8e78-956cca438bed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:19.629552 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f0446c7e-bf01-41b4-8e78-956cca438bed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:19.631252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1318/2601] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:19:19 2022] DELETE /identity/v3/projects/ac9ed3ea15414cd89d9a011496aa26ab => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:19.636489 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ce2b7-de8d-4dad-b6cc-1b52a8781e11 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:19.636965 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ce2b7-de8d-4dad-b6cc-1b52a8781e11 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:19.637101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ce2b7-de8d-4dad-b6cc-1b52a8781e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:19.661473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0c5ce2b7-de8d-4dad-b6cc-1b52a8781e11 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:19.661473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0c5ce2b7-de8d-4dad-b6cc-1b52a8781e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:19.716030 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1284/2602] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:19.723794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61bfbd-a767-4150-852f-5b78da6966d1 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:19.724089 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61bfbd-a767-4150-852f-5b78da6966d1 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:19.724369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61bfbd-a767-4150-852f-5b78da6966d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:19.742247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ed61bfbd-a767-4150-852f-5b78da6966d1 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:19.743469 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ed61bfbd-a767-4150-852f-5b78da6966d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:19.751752 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1319/2603] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:19.755641 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ce2b7-de8d-4dad-b6cc-1b52a8781e11 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:19.765543 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ce2b7-de8d-4dad-b6cc-1b52a8781e11 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:19.769979 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cf907-eda8-4952-8465-76e9aaaf15e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:19.770623 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cf907-eda8-4952-8465-76e9aaaf15e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:19.770908 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cf907-eda8-4952-8465-76e9aaaf15e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:19.771209 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cf907-eda8-4952-8465-76e9aaaf15e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:19.771933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cf907-eda8-4952-8465-76e9aaaf15e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:19:19.772238 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cf907-eda8-4952-8465-76e9aaaf15e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:19.773826 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cf907-eda8-4952-8465-76e9aaaf15e7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:19.785933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1285/2604] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:19:19.789915 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61bfbd-a767-4150-852f-5b78da6966d1 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:19.799697 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61bfbd-a767-4150-852f-5b78da6966d1 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:19.804401 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6c5db-e4ea-411e-9983-0d13e7384018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:19.805061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6c5db-e4ea-411e-9983-0d13e7384018 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:19.805351 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6c5db-e4ea-411e-9983-0d13e7384018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:19.805634 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6c5db-e4ea-411e-9983-0d13e7384018 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:19.806325 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6c5db-e4ea-411e-9983-0d13e7384018 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:19.808279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6c5db-e4ea-411e-9983-0d13e7384018 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:19.894980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1320/2605] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:19:19 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:19:19.899402 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cf907-eda8-4952-8465-76e9aaaf15e7 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:19.910501 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cf907-eda8-4952-8465-76e9aaaf15e7 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:19.936960 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:19.967147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2683894-c5c4-484b-982e-d31dc62e0996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:19.976422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2683894-c5c4-484b-982e-d31dc62e0996 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:19.976422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2683894-c5c4-484b-982e-d31dc62e0996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:19.976422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2683894-c5c4-484b-982e-d31dc62e0996 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:19.976422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2683894-c5c4-484b-982e-d31dc62e0996 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:19.976422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2683894-c5c4-484b-982e-d31dc62e0996 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:20.080638 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1286/2606] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:19:19 2022] POST /identity/v3/users => generated 282 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:19:20.084730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6c5db-e4ea-411e-9983-0d13e7384018 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:20.094896 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6c5db-e4ea-411e-9983-0d13e7384018 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:20.099654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c56a1-12d9-46b7-94db-32c0f839579e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:20.100316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c56a1-12d9-46b7-94db-32c0f839579e admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:20.100606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c56a1-12d9-46b7-94db-32c0f839579e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:20.100888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c56a1-12d9-46b7-94db-32c0f839579e admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:20.101613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c56a1-12d9-46b7-94db-32c0f839579e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:20.103327 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c56a1-12d9-46b7-94db-32c0f839579e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:20.114243 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1321/2607] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:20.118006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2683894-c5c4-484b-982e-d31dc62e0996 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:20.128296 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2683894-c5c4-484b-982e-d31dc62e0996 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:20.132871 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3c3ae-4f13-4f5e-9cde-3f39a2a3a364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:20.133511 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3c3ae-4f13-4f5e-9cde-3f39a2a3a364 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:20.133810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3c3ae-4f13-4f5e-9cde-3f39a2a3a364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:20.134088 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3c3ae-4f13-4f5e-9cde-3f39a2a3a364 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:20.134781 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3c3ae-4f13-4f5e-9cde-3f39a2a3a364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:20.136429 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3c3ae-4f13-4f5e-9cde-3f39a2a3a364 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:20.146156 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1287/2608] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:20.150383 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c56a1-12d9-46b7-94db-32c0f839579e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:20.160913 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c56a1-12d9-46b7-94db-32c0f839579e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:20.165482 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a525a8-4afc-4df2-a5c2-775bb3b5f128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:20.166362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a525a8-4afc-4df2-a5c2-775bb3b5f128 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:20.166645 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a525a8-4afc-4df2-a5c2-775bb3b5f128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:20.167489 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a525a8-4afc-4df2-a5c2-775bb3b5f128 admin admin] PATH_INFO: `/v3/projects/3268183ac3254198a3c45fed07383b91/users/0167bb8c408d4c61bb1fecdf8ce5d817/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:20.181702 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a525a8-4afc-4df2-a5c2-775bb3b5f128 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3268183ac3254198a3c45fed07383b91, user_id=0167bb8c408d4c61bb1fecdf8ce5d817, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:20.184962 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a525a8-4afc-4df2-a5c2-775bb3b5f128 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:20.298553 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1322/2609] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:19:20 2022] PUT /identity/v3/projects/3268183ac3254198a3c45fed07383b91/users/0167bb8c408d4c61bb1fecdf8ce5d817/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:20.310786 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392a07e-d77f-41f7-a0fa-e31003057cc9 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:20.311069 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392a07e-d77f-41f7-a0fa-e31003057cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:20.311371 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392a07e-d77f-41f7-a0fa-e31003057cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:20.337459 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4392a07e-d77f-41f7-a0fa-e31003057cc9 None None] MFA Rules not processed for user `0167bb8c408d4c61bb1fecdf8ce5d817`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:20.339308 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4392a07e-d77f-41f7-a0fa-e31003057cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:20.401006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1288/2610] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:20 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:20.406220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a525a8-4afc-4df2-a5c2-775bb3b5f128 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:20.467671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a525a8-4afc-4df2-a5c2-775bb3b5f128 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:20.472265 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2679bf-2626-49a3-a10c-0cf59ed55979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:20.472924 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2679bf-2626-49a3-a10c-0cf59ed55979 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:20.473207 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2679bf-2626-49a3-a10c-0cf59ed55979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:20.473484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2679bf-2626-49a3-a10c-0cf59ed55979 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:20.474180 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2679bf-2626-49a3-a10c-0cf59ed55979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:20.476132 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2679bf-2626-49a3-a10c-0cf59ed55979 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:20.562553 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1323/2611] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:19:20 2022] POST /identity/v3/projects => generated 363 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:19:20.566258 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392a07e-d77f-41f7-a0fa-e31003057cc9 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:20.577519 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392a07e-d77f-41f7-a0fa-e31003057cc9 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:20.630994 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7d2eb-342f-4051-918b-a978da88f5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:20.631678 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7d2eb-342f-4051-918b-a978da88f5f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:20.631966 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7d2eb-342f-4051-918b-a978da88f5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:20.632244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7d2eb-342f-4051-918b-a978da88f5f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:20.632931 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7d2eb-342f-4051-918b-a978da88f5f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:20.634846 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7d2eb-342f-4051-918b-a978da88f5f5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:20.743678 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1289/2612] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:19:20 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:19:20.747686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2679bf-2626-49a3-a10c-0cf59ed55979 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:20.758041 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2679bf-2626-49a3-a10c-0cf59ed55979 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:20.763062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa54c9-8cb3-4654-a557-c44291209abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:20.763761 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa54c9-8cb3-4654-a557-c44291209abb admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:20.764051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa54c9-8cb3-4654-a557-c44291209abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:20.764334 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa54c9-8cb3-4654-a557-c44291209abb admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:20.765061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa54c9-8cb3-4654-a557-c44291209abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:20.766667 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa54c9-8cb3-4654-a557-c44291209abb admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:20.778040 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1324/2613] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:20.781928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7d2eb-342f-4051-918b-a978da88f5f5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:20.792473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7d2eb-342f-4051-918b-a978da88f5f5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:20.797034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a54664-9e41-4f14-8176-25c3c436facc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:20.797902 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a54664-9e41-4f14-8176-25c3c436facc admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:20.798184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a54664-9e41-4f14-8176-25c3c436facc admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:20.798456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a54664-9e41-4f14-8176-25c3c436facc admin admin] PATH_INFO: `/v3/projects/320fcf9e4b594597ade3db6a320a8ab3/users/e6157856a140472ebe813ba514827e2e/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:20.812784 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a54664-9e41-4f14-8176-25c3c436facc admin admin] RBAC: Authorizing `identity:create_grant(project_id=320fcf9e4b594597ade3db6a320a8ab3, user_id=e6157856a140472ebe813ba514827e2e, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:20.815394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a54664-9e41-4f14-8176-25c3c436facc admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:20.880529 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1290/2614] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:19:20 2022] PUT /identity/v3/projects/320fcf9e4b594597ade3db6a320a8ab3/users/e6157856a140472ebe813ba514827e2e/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:20.884469 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa54c9-8cb3-4654-a557-c44291209abb admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:20.895269 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa54c9-8cb3-4654-a557-c44291209abb admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:20.915566 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:20.951563 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad26f05-4f4a-46d0-8cd6-95b7f43aa422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:20.952252 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad26f05-4f4a-46d0-8cd6-95b7f43aa422 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:20.952520 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad26f05-4f4a-46d0-8cd6-95b7f43aa422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:20.952803 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad26f05-4f4a-46d0-8cd6-95b7f43aa422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:20.953521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad26f05-4f4a-46d0-8cd6-95b7f43aa422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:20.955117 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad26f05-4f4a-46d0-8cd6-95b7f43aa422 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:20.965896 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1325/2615] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:20.969810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a54664-9e41-4f14-8176-25c3c436facc admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:20.975234 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:20.981034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a54664-9e41-4f14-8176-25c3c436facc admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:20.985904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565daf9-33f2-4fec-8823-9fdff29460ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:20.986783 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565daf9-33f2-4fec-8823-9fdff29460ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:20.987071 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565daf9-33f2-4fec-8823-9fdff29460ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:20.987381 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565daf9-33f2-4fec-8823-9fdff29460ff admin admin] PATH_INFO: `/v3/projects/320fcf9e4b594597ade3db6a320a8ab3/users/e6157856a140472ebe813ba514827e2e/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:20.989440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565daf9-33f2-4fec-8823-9fdff29460ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=320fcf9e4b594597ade3db6a320a8ab3, user_id=e6157856a140472ebe813ba514827e2e, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:20.992131 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565daf9-33f2-4fec-8823-9fdff29460ff admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:21.003194 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1291/2616] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:19:20 2022] PUT /identity/v3/projects/320fcf9e4b594597ade3db6a320a8ab3/users/e6157856a140472ebe813ba514827e2e/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:21.013592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225410f4-d26d-4792-bf28-00b83e9eef57 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:21.013888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225410f4-d26d-4792-bf28-00b83e9eef57 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:21.014171 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225410f4-d26d-4792-bf28-00b83e9eef57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:21.040503 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-225410f4-d26d-4792-bf28-00b83e9eef57 None None] MFA Rules not processed for user `e6157856a140472ebe813ba514827e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:21.042217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-225410f4-d26d-4792-bf28-00b83e9eef57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:21.108644 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1326/2617] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:21 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:21.120904 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565daf9-33f2-4fec-8823-9fdff29460ff admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:21.122559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e565daf9-33f2-4fec-8823-9fdff29460ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:21.179673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565daf9-33f2-4fec-8823-9fdff29460ff admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:21.183293 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154678ee-53a5-4e3a-b331-12704673aba6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:21.183929 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154678ee-53a5-4e3a-b331-12704673aba6 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:21.184216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154678ee-53a5-4e3a-b331-12704673aba6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:21.184497 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154678ee-53a5-4e3a-b331-12704673aba6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:21.193255 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154678ee-53a5-4e3a-b331-12704673aba6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:21.194676 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154678ee-53a5-4e3a-b331-12704673aba6 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:21.203974 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1292/2618] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:21 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:19:22.024631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225410f4-d26d-4792-bf28-00b83e9eef57 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:22.089014 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225410f4-d26d-4792-bf28-00b83e9eef57 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:22.093948 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bd09f-b0ef-4e7a-894d-5343421314d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:22.094830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bd09f-b0ef-4e7a-894d-5343421314d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:22.095115 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bd09f-b0ef-4e7a-894d-5343421314d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:22.095422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bd09f-b0ef-4e7a-894d-5343421314d1 admin admin] PATH_INFO: `/v3/users/0167bb8c408d4c61bb1fecdf8ce5d817` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:22.096625 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bd09f-b0ef-4e7a-894d-5343421314d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0167bb8c408d4c61bb1fecdf8ce5d817)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:22.098572 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bd09f-b0ef-4e7a-894d-5343421314d1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:22.236057 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fc9bd09f-b0ef-4e7a-894d-5343421314d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0167bb8c408d4c61bb1fecdf8ce5d817'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:22.236700 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fc9bd09f-b0ef-4e7a-894d-5343421314d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0167bb8c408d4c61bb1fecdf8ce5d817'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:22.237799 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:22.249290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fc9bd09f-b0ef-4e7a-894d-5343421314d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0167bb8c408d4c61bb1fecdf8ce5d817'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:22.249919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fc9bd09f-b0ef-4e7a-894d-5343421314d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0167bb8c408d4c61bb1fecdf8ce5d817'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:22.256500 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-fc9bd09f-b0ef-4e7a-894d-5343421314d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0167bb8c408d4c61bb1fecdf8ce5d817'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:22.320370 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1327/2619] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:19:22 2022] DELETE /identity/v3/users/0167bb8c408d4c61bb1fecdf8ce5d817 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:22.325403 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a5d20-9bf9-4601-879b-a5a52bddd71f None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:22.325833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a5d20-9bf9-4601-879b-a5a52bddd71f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:22.325967 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a5d20-9bf9-4601-879b-a5a52bddd71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:22.344145 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b03a5d20-9bf9-4601-879b-a5a52bddd71f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:22.345357 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b03a5d20-9bf9-4601-879b-a5a52bddd71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:22.402948 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1293/2620] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:22.412593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bd09f-b0ef-4e7a-894d-5343421314d1 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:22.414009 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-fc9bd09f-b0ef-4e7a-894d-5343421314d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:22.470347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bd09f-b0ef-4e7a-894d-5343421314d1 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:22.474042 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09ca31-77a6-4d39-851c-bb51a61c2d8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:22.474685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09ca31-77a6-4d39-851c-bb51a61c2d8e service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:22.474971 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09ca31-77a6-4d39-851c-bb51a61c2d8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:22.475275 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09ca31-77a6-4d39-851c-bb51a61c2d8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:22.484166 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce09ca31-77a6-4d39-851c-bb51a61c2d8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:22.485598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce09ca31-77a6-4d39-851c-bb51a61c2d8e service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:22.495297 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1328/2621] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:19:22.997606 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a5d20-9bf9-4601-879b-a5a52bddd71f None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:22.999106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b03a5d20-9bf9-4601-879b-a5a52bddd71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:23.013046 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a5d20-9bf9-4601-879b-a5a52bddd71f None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:23.018220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88054dec-fef6-4595-848f-7108600c97ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:23.019094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88054dec-fef6-4595-848f-7108600c97ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:23.019414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88054dec-fef6-4595-848f-7108600c97ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:23.019695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88054dec-fef6-4595-848f-7108600c97ff admin admin] PATH_INFO: `/v3/projects/3268183ac3254198a3c45fed07383b91` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:23.020882 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88054dec-fef6-4595-848f-7108600c97ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=3268183ac3254198a3c45fed07383b91)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:23.022853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88054dec-fef6-4595-848f-7108600c97ff admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:23.140689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-88054dec-fef6-4595-848f-7108600c97ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3268183ac3254198a3c45fed07383b91', 'request_id': 'req-88054dec-fef6-4595-848f-7108600c97ff'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:23.223599 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-88054dec-fef6-4595-848f-7108600c97ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3268183ac3254198a3c45fed07383b91', 'request_id': 'req-88054dec-fef6-4595-848f-7108600c97ff'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:23.230347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-88054dec-fef6-4595-848f-7108600c97ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3268183ac3254198a3c45fed07383b91', 'request_id': 'req-88054dec-fef6-4595-848f-7108600c97ff'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:23.242228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-88054dec-fef6-4595-848f-7108600c97ff admin admin] The token cache is being invalidate because project 3268183ac3254198a3c45fed07383b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:19:23.242525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-88054dec-fef6-4595-848f-7108600c97ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:23.243226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-88054dec-fef6-4595-848f-7108600c97ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:23.245404 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1294/2622] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:19:22 2022] DELETE /identity/v3/projects/3268183ac3254198a3c45fed07383b91 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:23.249337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09ca31-77a6-4d39-851c-bb51a61c2d8e service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:23.259279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09ca31-77a6-4d39-851c-bb51a61c2d8e service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:23.314266 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a45b1d-d45b-4b40-a352-a49b8dbf9b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:23.315129 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a45b1d-d45b-4b40-a352-a49b8dbf9b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:23.315447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a45b1d-d45b-4b40-a352-a49b8dbf9b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:23.315723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a45b1d-d45b-4b40-a352-a49b8dbf9b46 admin admin] PATH_INFO: `/v3/users/e6157856a140472ebe813ba514827e2e` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:23.316902 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a45b1d-d45b-4b40-a352-a49b8dbf9b46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6157856a140472ebe813ba514827e2e)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:23.318826 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a45b1d-d45b-4b40-a352-a49b8dbf9b46 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:23.447618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-48a45b1d-d45b-4b40-a352-a49b8dbf9b46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6157856a140472ebe813ba514827e2e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:23.448752 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-48a45b1d-d45b-4b40-a352-a49b8dbf9b46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6157856a140472ebe813ba514827e2e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:23.460317 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-48a45b1d-d45b-4b40-a352-a49b8dbf9b46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6157856a140472ebe813ba514827e2e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:23.460973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-48a45b1d-d45b-4b40-a352-a49b8dbf9b46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6157856a140472ebe813ba514827e2e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:23.466211 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:23.468745 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-48a45b1d-d45b-4b40-a352-a49b8dbf9b46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6157856a140472ebe813ba514827e2e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:23.564903 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1329/2623] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:19:23 2022] DELETE /identity/v3/users/e6157856a140472ebe813ba514827e2e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:24.060171 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88054dec-fef6-4595-848f-7108600c97ff admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:24.064023 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-88054dec-fef6-4595-848f-7108600c97ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:24.074624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88054dec-fef6-4595-848f-7108600c97ff admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:24.087038 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:24.131009 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18aa41b-95d7-412d-8605-078d5ab23da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:24.131899 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18aa41b-95d7-412d-8605-078d5ab23da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:24.132186 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18aa41b-95d7-412d-8605-078d5ab23da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:24.132465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18aa41b-95d7-412d-8605-078d5ab23da7 admin admin] PATH_INFO: `/v3/projects/320fcf9e4b594597ade3db6a320a8ab3` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:24.133633 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18aa41b-95d7-412d-8605-078d5ab23da7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=320fcf9e4b594597ade3db6a320a8ab3)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:24.135618 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18aa41b-95d7-412d-8605-078d5ab23da7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:24.235212 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e18aa41b-95d7-412d-8605-078d5ab23da7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '320fcf9e4b594597ade3db6a320a8ab3', 'request_id': 'req-e18aa41b-95d7-412d-8605-078d5ab23da7'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:24.303433 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e18aa41b-95d7-412d-8605-078d5ab23da7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '320fcf9e4b594597ade3db6a320a8ab3', 'request_id': 'req-e18aa41b-95d7-412d-8605-078d5ab23da7'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:24.309255 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e18aa41b-95d7-412d-8605-078d5ab23da7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '320fcf9e4b594597ade3db6a320a8ab3', 'request_id': 'req-e18aa41b-95d7-412d-8605-078d5ab23da7'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:24.320461 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e18aa41b-95d7-412d-8605-078d5ab23da7 admin admin] The token cache is being invalidate because project 320fcf9e4b594597ade3db6a320a8ab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:19:24.320749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e18aa41b-95d7-412d-8605-078d5ab23da7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:24.321378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e18aa41b-95d7-412d-8605-078d5ab23da7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:24.322938 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1295/2624] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:19:24 2022] DELETE /identity/v3/projects/320fcf9e4b594597ade3db6a320a8ab3 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:24.328071 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419384c-6bc9-4314-8a84-dd27c8cc2ca6 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:24.328524 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419384c-6bc9-4314-8a84-dd27c8cc2ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:24.328637 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419384c-6bc9-4314-8a84-dd27c8cc2ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:24.347621 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0419384c-6bc9-4314-8a84-dd27c8cc2ca6 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:24.348820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0419384c-6bc9-4314-8a84-dd27c8cc2ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:24.408500 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1330/2625] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:24.416304 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6ceb2-3827-4500-af06-41ade8442313 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:24.416599 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6ceb2-3827-4500-af06-41ade8442313 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:24.416879 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6ceb2-3827-4500-af06-41ade8442313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:24.436036 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-9bf6ceb2-3827-4500-af06-41ade8442313 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:24.437744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9bf6ceb2-3827-4500-af06-41ade8442313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:24.445310 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1296/2626] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:24.450009 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419384c-6bc9-4314-8a84-dd27c8cc2ca6 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:24.459890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419384c-6bc9-4314-8a84-dd27c8cc2ca6 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:24.464484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481740b3-55db-4a78-9a1d-6ffca53e4412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:24.465167 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481740b3-55db-4a78-9a1d-6ffca53e4412 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:24.465454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481740b3-55db-4a78-9a1d-6ffca53e4412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:24.465738 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481740b3-55db-4a78-9a1d-6ffca53e4412 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:24.466462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481740b3-55db-4a78-9a1d-6ffca53e4412 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:19:24.466770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481740b3-55db-4a78-9a1d-6ffca53e4412 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:24.468432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481740b3-55db-4a78-9a1d-6ffca53e4412 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:24.480149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1331/2627] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:19:24.484432 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6ceb2-3827-4500-af06-41ade8442313 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:24.493748 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6ceb2-3827-4500-af06-41ade8442313 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:24.499001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347bbe0-d6fb-4e86-bbad-8c384d027233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:24.499677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347bbe0-d6fb-4e86-bbad-8c384d027233 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:24.499963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347bbe0-d6fb-4e86-bbad-8c384d027233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:24.500236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347bbe0-d6fb-4e86-bbad-8c384d027233 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:24.500928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a347bbe0-d6fb-4e86-bbad-8c384d027233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:24.502899 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a347bbe0-d6fb-4e86-bbad-8c384d027233 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:24.571620 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1297/2628] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:19:24 2022] POST /identity/v3/projects => generated 389 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:19:24.575712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481740b3-55db-4a78-9a1d-6ffca53e4412 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:24.585621 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481740b3-55db-4a78-9a1d-6ffca53e4412 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:24.641023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3a0b5-c9ca-47f3-aa2a-cae85508bfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:24.641691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3a0b5-c9ca-47f3-aa2a-cae85508bfd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:24.641981 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3a0b5-c9ca-47f3-aa2a-cae85508bfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:24.642265 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3a0b5-c9ca-47f3-aa2a-cae85508bfd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:24.642956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3a0b5-c9ca-47f3-aa2a-cae85508bfd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:24.644915 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3a0b5-c9ca-47f3-aa2a-cae85508bfd3 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:24.714506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1332/2629] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:19:24 2022] POST /identity/v3/users => generated 295 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:19:24.718555 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347bbe0-d6fb-4e86-bbad-8c384d027233 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:24.728894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347bbe0-d6fb-4e86-bbad-8c384d027233 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:24.734160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c28843-c1ab-4dc6-a58e-921c54243713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:24.734812 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c28843-c1ab-4dc6-a58e-921c54243713 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:24.735097 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c28843-c1ab-4dc6-a58e-921c54243713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:24.735401 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c28843-c1ab-4dc6-a58e-921c54243713 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:24.736213 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c28843-c1ab-4dc6-a58e-921c54243713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:24.737847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c28843-c1ab-4dc6-a58e-921c54243713 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:24.748771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1298/2630] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:24.752517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3a0b5-c9ca-47f3-aa2a-cae85508bfd3 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:24.762329 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3a0b5-c9ca-47f3-aa2a-cae85508bfd3 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:24.766910 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36540fd4-d8cb-4abe-b795-8e900eefdf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:24.767592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36540fd4-d8cb-4abe-b795-8e900eefdf51 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:24.767880 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36540fd4-d8cb-4abe-b795-8e900eefdf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:24.768158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36540fd4-d8cb-4abe-b795-8e900eefdf51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:24.768881 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36540fd4-d8cb-4abe-b795-8e900eefdf51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:24.770496 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36540fd4-d8cb-4abe-b795-8e900eefdf51 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:24.780402 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1333/2631] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:24.784328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c28843-c1ab-4dc6-a58e-921c54243713 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:24.794540 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c28843-c1ab-4dc6-a58e-921c54243713 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:24.799362 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701b504-e992-4934-abf6-9121acb5cfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:24.800252 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701b504-e992-4934-abf6-9121acb5cfc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:24.800562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701b504-e992-4934-abf6-9121acb5cfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:24.800840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701b504-e992-4934-abf6-9121acb5cfc5 admin admin] PATH_INFO: `/v3/projects/1c7ba8d8b0284506844d3dd9a8462156/users/dd3ac8f5ccd740aaa8bd41faf5c6cbbd/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:24.816263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2701b504-e992-4934-abf6-9121acb5cfc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7ba8d8b0284506844d3dd9a8462156, user_id=dd3ac8f5ccd740aaa8bd41faf5c6cbbd, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:24.818950 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2701b504-e992-4934-abf6-9121acb5cfc5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:24.909575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1299/2632] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:19:24 2022] PUT /identity/v3/projects/1c7ba8d8b0284506844d3dd9a8462156/users/dd3ac8f5ccd740aaa8bd41faf5c6cbbd/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:24.921256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4432a-e545-4357-aa89-3a73cf299f20 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:24.921557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4432a-e545-4357-aa89-3a73cf299f20 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:24.921843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4432a-e545-4357-aa89-3a73cf299f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:24.949854 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-20c4432a-e545-4357-aa89-3a73cf299f20 None None] MFA Rules not processed for user `dd3ac8f5ccd740aaa8bd41faf5c6cbbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:24.951064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-20c4432a-e545-4357-aa89-3a73cf299f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:24.991260 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:25.015036 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1334/2633] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:25.019537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701b504-e992-4934-abf6-9121acb5cfc5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:25.063883 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:25.080729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701b504-e992-4934-abf6-9121acb5cfc5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:25.085672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c3168-01b7-434a-a7fc-55b5234b6f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:25.086347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c3168-01b7-434a-a7fc-55b5234b6f6a admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:25.086630 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c3168-01b7-434a-a7fc-55b5234b6f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:25.086910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c3168-01b7-434a-a7fc-55b5234b6f6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:25.087624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c3168-01b7-434a-a7fc-55b5234b6f6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:25.090148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c3168-01b7-434a-a7fc-55b5234b6f6a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:25.177329 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1300/2634] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:19:25 2022] POST /identity/v3/projects => generated 389 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:19:25.181308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4432a-e545-4357-aa89-3a73cf299f20 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:25.192149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4432a-e545-4357-aa89-3a73cf299f20 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:25.247555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb33fe2-7d51-4f6c-9940-95a462abbc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:25.248246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb33fe2-7d51-4f6c-9940-95a462abbc3c admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:25.248532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb33fe2-7d51-4f6c-9940-95a462abbc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:25.248810 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb33fe2-7d51-4f6c-9940-95a462abbc3c admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:25.249485 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb33fe2-7d51-4f6c-9940-95a462abbc3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:25.251421 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb33fe2-7d51-4f6c-9940-95a462abbc3c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:25.332025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1335/2635] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:19:25 2022] POST /identity/v3/users => generated 301 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:19:25.336005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c3168-01b7-434a-a7fc-55b5234b6f6a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:25.346651 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c3168-01b7-434a-a7fc-55b5234b6f6a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:25.351354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae6b9d-8a3e-429c-84a6-cd78481a3990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:25.358237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae6b9d-8a3e-429c-84a6-cd78481a3990 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:25.358237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae6b9d-8a3e-429c-84a6-cd78481a3990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:25.358237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae6b9d-8a3e-429c-84a6-cd78481a3990 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:25.358237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae6b9d-8a3e-429c-84a6-cd78481a3990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:25.358237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae6b9d-8a3e-429c-84a6-cd78481a3990 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:25.365738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1301/2636] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:25.370528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb33fe2-7d51-4f6c-9940-95a462abbc3c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:25.380492 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb33fe2-7d51-4f6c-9940-95a462abbc3c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:25.385062 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8301a8-5de1-4eec-89d5-527d1fcdeee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:25.386035 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8301a8-5de1-4eec-89d5-527d1fcdeee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:25.386322 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8301a8-5de1-4eec-89d5-527d1fcdeee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:25.386600 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8301a8-5de1-4eec-89d5-527d1fcdeee8 admin admin] PATH_INFO: `/v3/projects/5a54592abecd4a19919a22347153410e/users/e26fba49fd18470ab4f28a33f2100a74/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:25.400875 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8301a8-5de1-4eec-89d5-527d1fcdeee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a54592abecd4a19919a22347153410e, user_id=e26fba49fd18470ab4f28a33f2100a74, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:25.403897 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8301a8-5de1-4eec-89d5-527d1fcdeee8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:25.550082 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1336/2637] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:19:25 2022] PUT /identity/v3/projects/5a54592abecd4a19919a22347153410e/users/e26fba49fd18470ab4f28a33f2100a74/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:19:25.553955 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae6b9d-8a3e-429c-84a6-cd78481a3990 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:25.564482 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae6b9d-8a3e-429c-84a6-cd78481a3990 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:25.619254 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbb19d-9145-4268-957c-7608221e85c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:25.620520 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbb19d-9145-4268-957c-7608221e85c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:25.620806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbb19d-9145-4268-957c-7608221e85c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:25.621083 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbb19d-9145-4268-957c-7608221e85c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:25.621819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbb19d-9145-4268-957c-7608221e85c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:25.623428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbb19d-9145-4268-957c-7608221e85c3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:25.632896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1302/2638] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:25.637111 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8301a8-5de1-4eec-89d5-527d1fcdeee8 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:25.647384 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8301a8-5de1-4eec-89d5-527d1fcdeee8 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:25.652791 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965f015-91f8-4f8f-9282-0250c457cacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:25.653703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965f015-91f8-4f8f-9282-0250c457cacf admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:25.654003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965f015-91f8-4f8f-9282-0250c457cacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:25.654273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965f015-91f8-4f8f-9282-0250c457cacf admin admin] PATH_INFO: `/v3/projects/5a54592abecd4a19919a22347153410e/users/e26fba49fd18470ab4f28a33f2100a74/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:25.656329 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965f015-91f8-4f8f-9282-0250c457cacf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a54592abecd4a19919a22347153410e, user_id=e26fba49fd18470ab4f28a33f2100a74, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:25.658978 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965f015-91f8-4f8f-9282-0250c457cacf admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:25.669703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1337/2639] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:19:25 2022] PUT /identity/v3/projects/5a54592abecd4a19919a22347153410e/users/e26fba49fd18470ab4f28a33f2100a74/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:25.680094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14cdea-0425-46cf-9c76-60beb4792874 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:25.680421 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14cdea-0425-46cf-9c76-60beb4792874 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:25.680734 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14cdea-0425-46cf-9c76-60beb4792874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:25.708426 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-2e14cdea-0425-46cf-9c76-60beb4792874 None None] MFA Rules not processed for user `e26fba49fd18470ab4f28a33f2100a74`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:25.709637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2e14cdea-0425-46cf-9c76-60beb4792874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:25.776663 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1303/2640] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:25.787586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965f015-91f8-4f8f-9282-0250c457cacf admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:25.789256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9965f015-91f8-4f8f-9282-0250c457cacf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:25.845843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965f015-91f8-4f8f-9282-0250c457cacf admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:25.849489 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce15d2-0bcd-48b8-9719-fde5ef6dcb72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:25.850138 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce15d2-0bcd-48b8-9719-fde5ef6dcb72 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:25.850429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce15d2-0bcd-48b8-9719-fde5ef6dcb72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:25.850715 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce15d2-0bcd-48b8-9719-fde5ef6dcb72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:25.860608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce15d2-0bcd-48b8-9719-fde5ef6dcb72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:25.862071 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce15d2-0bcd-48b8-9719-fde5ef6dcb72 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:25.872642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1338/2641] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:19:26.003831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14cdea-0425-46cf-9c76-60beb4792874 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:26.044673 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:26.065385 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14cdea-0425-46cf-9c76-60beb4792874 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:26.070012 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436677f4-1e0d-4946-b4c3-29b7857a1d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:26.070877 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436677f4-1e0d-4946-b4c3-29b7857a1d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:26.071186 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436677f4-1e0d-4946-b4c3-29b7857a1d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:26.071473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436677f4-1e0d-4946-b4c3-29b7857a1d06 admin admin] PATH_INFO: `/v3/users/dd3ac8f5ccd740aaa8bd41faf5c6cbbd` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:26.072663 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436677f4-1e0d-4946-b4c3-29b7857a1d06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd3ac8f5ccd740aaa8bd41faf5c6cbbd)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:26.074581 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436677f4-1e0d-4946-b4c3-29b7857a1d06 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:26.248146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-436677f4-1e0d-4946-b4c3-29b7857a1d06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd3ac8f5ccd740aaa8bd41faf5c6cbbd'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:26.254857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-436677f4-1e0d-4946-b4c3-29b7857a1d06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd3ac8f5ccd740aaa8bd41faf5c6cbbd'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:26.255526 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-436677f4-1e0d-4946-b4c3-29b7857a1d06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd3ac8f5ccd740aaa8bd41faf5c6cbbd'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:26.256180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-436677f4-1e0d-4946-b4c3-29b7857a1d06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd3ac8f5ccd740aaa8bd41faf5c6cbbd'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:26.268154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-436677f4-1e0d-4946-b4c3-29b7857a1d06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd3ac8f5ccd740aaa8bd41faf5c6cbbd'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:26.337585 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1304/2642] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:19:26 2022] DELETE /identity/v3/users/dd3ac8f5ccd740aaa8bd41faf5c6cbbd => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:26.342418 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4239f4-d223-417a-b362-1913846a15f0 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:26.342905 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4239f4-d223-417a-b362-1913846a15f0 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:26.343012 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4239f4-d223-417a-b362-1913846a15f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:26.361908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-aa4239f4-d223-417a-b362-1913846a15f0 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:26.363104 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-aa4239f4-d223-417a-b362-1913846a15f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:26.394263 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:26.422002 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1339/2643] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:26.431575 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436677f4-1e0d-4946-b4c3-29b7857a1d06 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:26.434404 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-436677f4-1e0d-4946-b4c3-29b7857a1d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:26.489435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436677f4-1e0d-4946-b4c3-29b7857a1d06 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:26.493109 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5961a-c3ef-4150-8c64-3fef49609ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:26.493751 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5961a-c3ef-4150-8c64-3fef49609ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:26.494037 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5961a-c3ef-4150-8c64-3fef49609ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:26.494356 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5961a-c3ef-4150-8c64-3fef49609ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:26.503345 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5961a-c3ef-4150-8c64-3fef49609ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:26.504725 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5961a-c3ef-4150-8c64-3fef49609ea3 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:26.514045 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1305/2644] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:19:26.917143 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4239f4-d223-417a-b362-1913846a15f0 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:26.918711 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-aa4239f4-d223-417a-b362-1913846a15f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:26.932945 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4239f4-d223-417a-b362-1913846a15f0 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:26.937871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4c8b7-1f22-42ae-b6ab-bb2a9a4abedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:26.938762 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4c8b7-1f22-42ae-b6ab-bb2a9a4abedb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:26.939048 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4c8b7-1f22-42ae-b6ab-bb2a9a4abedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:26.939362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4c8b7-1f22-42ae-b6ab-bb2a9a4abedb admin admin] PATH_INFO: `/v3/projects/1c7ba8d8b0284506844d3dd9a8462156` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:26.940538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4c8b7-1f22-42ae-b6ab-bb2a9a4abedb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c7ba8d8b0284506844d3dd9a8462156)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:26.942462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4c8b7-1f22-42ae-b6ab-bb2a9a4abedb admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:27.040266 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6bc4c8b7-1f22-42ae-b6ab-bb2a9a4abedb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c7ba8d8b0284506844d3dd9a8462156', 'request_id': 'req-6bc4c8b7-1f22-42ae-b6ab-bb2a9a4abedb'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:27.103521 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6bc4c8b7-1f22-42ae-b6ab-bb2a9a4abedb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c7ba8d8b0284506844d3dd9a8462156', 'request_id': 'req-6bc4c8b7-1f22-42ae-b6ab-bb2a9a4abedb'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:27.109808 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6bc4c8b7-1f22-42ae-b6ab-bb2a9a4abedb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c7ba8d8b0284506844d3dd9a8462156', 'request_id': 'req-6bc4c8b7-1f22-42ae-b6ab-bb2a9a4abedb'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:27.121747 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6bc4c8b7-1f22-42ae-b6ab-bb2a9a4abedb admin admin] The token cache is being invalidate because project 1c7ba8d8b0284506844d3dd9a8462156 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:19:27.122038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6bc4c8b7-1f22-42ae-b6ab-bb2a9a4abedb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:27.122695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6bc4c8b7-1f22-42ae-b6ab-bb2a9a4abedb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:27.124275 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1340/2645] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:19:26 2022] DELETE /identity/v3/projects/1c7ba8d8b0284506844d3dd9a8462156 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:27.128337 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5961a-c3ef-4150-8c64-3fef49609ea3 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:27.137820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5961a-c3ef-4150-8c64-3fef49609ea3 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:27.193005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059e939-f494-4740-b5f8-aa8d191eb32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:27.193866 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059e939-f494-4740-b5f8-aa8d191eb32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:27.194153 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059e939-f494-4740-b5f8-aa8d191eb32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:27.194435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059e939-f494-4740-b5f8-aa8d191eb32e admin admin] PATH_INFO: `/v3/users/e26fba49fd18470ab4f28a33f2100a74` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:27.195645 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059e939-f494-4740-b5f8-aa8d191eb32e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e26fba49fd18470ab4f28a33f2100a74)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:27.197571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059e939-f494-4740-b5f8-aa8d191eb32e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:27.199044 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:27.360981 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0059e939-f494-4740-b5f8-aa8d191eb32e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e26fba49fd18470ab4f28a33f2100a74'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:27.367884 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0059e939-f494-4740-b5f8-aa8d191eb32e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e26fba49fd18470ab4f28a33f2100a74'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:27.368538 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0059e939-f494-4740-b5f8-aa8d191eb32e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e26fba49fd18470ab4f28a33f2100a74'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:27.369142 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0059e939-f494-4740-b5f8-aa8d191eb32e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e26fba49fd18470ab4f28a33f2100a74'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:27.381272 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0059e939-f494-4740-b5f8-aa8d191eb32e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e26fba49fd18470ab4f28a33f2100a74'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:27.452054 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1306/2646] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:19:27 2022] DELETE /identity/v3/users/e26fba49fd18470ab4f28a33f2100a74 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:27.819282 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4c8b7-1f22-42ae-b6ab-bb2a9a4abedb admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:27.820771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6bc4c8b7-1f22-42ae-b6ab-bb2a9a4abedb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:27.834369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4c8b7-1f22-42ae-b6ab-bb2a9a4abedb admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:27.893604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a44621-3bec-41bc-b369-1372eaf8ee96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:27.894416 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a44621-3bec-41bc-b369-1372eaf8ee96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:27.894709 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a44621-3bec-41bc-b369-1372eaf8ee96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:27.894993 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a44621-3bec-41bc-b369-1372eaf8ee96 admin admin] PATH_INFO: `/v3/projects/5a54592abecd4a19919a22347153410e` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:27.896226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a44621-3bec-41bc-b369-1372eaf8ee96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a54592abecd4a19919a22347153410e)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:27.898148 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a44621-3bec-41bc-b369-1372eaf8ee96 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:28.008524 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b6a44621-3bec-41bc-b369-1372eaf8ee96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a54592abecd4a19919a22347153410e', 'request_id': 'req-b6a44621-3bec-41bc-b369-1372eaf8ee96'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:28.103704 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b6a44621-3bec-41bc-b369-1372eaf8ee96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a54592abecd4a19919a22347153410e', 'request_id': 'req-b6a44621-3bec-41bc-b369-1372eaf8ee96'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:28.111375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:28.112885 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b6a44621-3bec-41bc-b369-1372eaf8ee96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a54592abecd4a19919a22347153410e', 'request_id': 'req-b6a44621-3bec-41bc-b369-1372eaf8ee96'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:28.124264 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b6a44621-3bec-41bc-b369-1372eaf8ee96 admin admin] The token cache is being invalidate because project 5a54592abecd4a19919a22347153410e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:19:28.124556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b6a44621-3bec-41bc-b369-1372eaf8ee96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:28.125240 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b6a44621-3bec-41bc-b369-1372eaf8ee96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:28.126783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1341/2647] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:19:27 2022] DELETE /identity/v3/projects/5a54592abecd4a19919a22347153410e => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:28.131743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32096bf4-e20d-4476-b7ea-dd75cc61fa8c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:28.132224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32096bf4-e20d-4476-b7ea-dd75cc61fa8c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:28.132330 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32096bf4-e20d-4476-b7ea-dd75cc61fa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:28.150354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-32096bf4-e20d-4476-b7ea-dd75cc61fa8c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:28.152281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-32096bf4-e20d-4476-b7ea-dd75cc61fa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:28.209737 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1307/2648] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:28.217985 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea798458-3ecc-466a-b014-3a2e3af484d3 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:28.218280 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea798458-3ecc-466a-b014-3a2e3af484d3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:28.219635 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea798458-3ecc-466a-b014-3a2e3af484d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:28.237315 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ea798458-3ecc-466a-b014-3a2e3af484d3 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:28.238526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ea798458-3ecc-466a-b014-3a2e3af484d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:28.246919 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1342/2649] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:28.251057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32096bf4-e20d-4476-b7ea-dd75cc61fa8c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:28.261381 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32096bf4-e20d-4476-b7ea-dd75cc61fa8c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:28.266092 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c3325-92dc-4c80-a6f6-8e233e77af17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:28.266737 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c3325-92dc-4c80-a6f6-8e233e77af17 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:28.267024 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c3325-92dc-4c80-a6f6-8e233e77af17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:28.267322 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c3325-92dc-4c80-a6f6-8e233e77af17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:28.268071 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c3325-92dc-4c80-a6f6-8e233e77af17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:19:28.268374 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c3325-92dc-4c80-a6f6-8e233e77af17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:28.269968 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c3325-92dc-4c80-a6f6-8e233e77af17 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:28.281856 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1308/2650] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:19:28.286105 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea798458-3ecc-466a-b014-3a2e3af484d3 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:28.295758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea798458-3ecc-466a-b014-3a2e3af484d3 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:28.300829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5298a-28fa-48b2-b0e7-6f33d4872a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:28.301472 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5298a-28fa-48b2-b0e7-6f33d4872a38 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:28.301760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5298a-28fa-48b2-b0e7-6f33d4872a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:28.302041 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5298a-28fa-48b2-b0e7-6f33d4872a38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:28.302794 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5298a-28fa-48b2-b0e7-6f33d4872a38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:28.304755 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5298a-28fa-48b2-b0e7-6f33d4872a38 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:28.389184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1343/2651] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:19:28 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:19:28.393219 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c3325-92dc-4c80-a6f6-8e233e77af17 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:28.404180 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c3325-92dc-4c80-a6f6-8e233e77af17 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:28.459584 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80110645-99f6-44fc-bfd9-097d27f63d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:28.460244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80110645-99f6-44fc-bfd9-097d27f63d3a admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:28.460530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80110645-99f6-44fc-bfd9-097d27f63d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:28.460806 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80110645-99f6-44fc-bfd9-097d27f63d3a admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:28.461478 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80110645-99f6-44fc-bfd9-097d27f63d3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:28.463427 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80110645-99f6-44fc-bfd9-097d27f63d3a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:28.472698 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:28.548896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1309/2652] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:19:28 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:19:28.552947 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5298a-28fa-48b2-b0e7-6f33d4872a38 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:28.564038 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5298a-28fa-48b2-b0e7-6f33d4872a38 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:28.568627 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d67ca5-da86-4636-8f8d-05ebb2d5a2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:28.569284 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d67ca5-da86-4636-8f8d-05ebb2d5a2bb admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:28.569570 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d67ca5-da86-4636-8f8d-05ebb2d5a2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:28.569844 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d67ca5-da86-4636-8f8d-05ebb2d5a2bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:28.570579 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d67ca5-da86-4636-8f8d-05ebb2d5a2bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:28.572217 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d67ca5-da86-4636-8f8d-05ebb2d5a2bb admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:28.583158 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1344/2653] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:28.587077 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80110645-99f6-44fc-bfd9-097d27f63d3a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:28.597520 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80110645-99f6-44fc-bfd9-097d27f63d3a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:28.602102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b40a6-869a-41ba-8b02-fed9f7485b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:28.602755 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b40a6-869a-41ba-8b02-fed9f7485b18 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:28.603043 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b40a6-869a-41ba-8b02-fed9f7485b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:28.603363 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b40a6-869a-41ba-8b02-fed9f7485b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:28.604275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b40a6-869a-41ba-8b02-fed9f7485b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:28.606207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b40a6-869a-41ba-8b02-fed9f7485b18 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:28.616091 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1310/2654] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:28.619972 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d67ca5-da86-4636-8f8d-05ebb2d5a2bb admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:28.629770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d67ca5-da86-4636-8f8d-05ebb2d5a2bb admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:28.634381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c53b4e-896c-4d86-833e-83f05582e974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:28.635798 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c53b4e-896c-4d86-833e-83f05582e974 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:28.636087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c53b4e-896c-4d86-833e-83f05582e974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:28.636365 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c53b4e-896c-4d86-833e-83f05582e974 admin admin] PATH_INFO: `/v3/projects/c60eae7819eb4198adefa1df3616f2c3/users/bba2c9a3a5ec46eea3e8e5467f34d92b/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:28.650510 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c53b4e-896c-4d86-833e-83f05582e974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60eae7819eb4198adefa1df3616f2c3, user_id=bba2c9a3a5ec46eea3e8e5467f34d92b, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:28.654110 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c53b4e-896c-4d86-833e-83f05582e974 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:28.714593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1345/2655] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:19:28 2022] PUT /identity/v3/projects/c60eae7819eb4198adefa1df3616f2c3/users/bba2c9a3a5ec46eea3e8e5467f34d92b/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:28.725504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be02a5b-b0cd-4d89-aa44-2c48a794a6d0 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:28.733550 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be02a5b-b0cd-4d89-aa44-2c48a794a6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:28.733550 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be02a5b-b0cd-4d89-aa44-2c48a794a6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:28.752960 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-1be02a5b-b0cd-4d89-aa44-2c48a794a6d0 None None] MFA Rules not processed for user `bba2c9a3a5ec46eea3e8e5467f34d92b`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:28.754163 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1be02a5b-b0cd-4d89-aa44-2c48a794a6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:28.816880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1311/2656] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:28.821277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c53b4e-896c-4d86-833e-83f05582e974 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:28.881544 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c53b4e-896c-4d86-833e-83f05582e974 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:28.886910 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cca2e6-3369-4997-8e74-e4e981aa683c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:28.887594 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cca2e6-3369-4997-8e74-e4e981aa683c admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:28.887886 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cca2e6-3369-4997-8e74-e4e981aa683c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:28.888167 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cca2e6-3369-4997-8e74-e4e981aa683c admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:28.888868 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cca2e6-3369-4997-8e74-e4e981aa683c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:28.890767 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cca2e6-3369-4997-8e74-e4e981aa683c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:28.994648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1346/2657] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:19:28 2022] POST /identity/v3/projects => generated 379 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:19:28.998320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be02a5b-b0cd-4d89-aa44-2c48a794a6d0 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:29.010027 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be02a5b-b0cd-4d89-aa44-2c48a794a6d0 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:29.065277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5fae61-fb05-4da2-a7a0-53cbdb455888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:29.065948 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5fae61-fb05-4da2-a7a0-53cbdb455888 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:29.066233 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5fae61-fb05-4da2-a7a0-53cbdb455888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:29.066514 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5fae61-fb05-4da2-a7a0-53cbdb455888 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:29.067234 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5fae61-fb05-4da2-a7a0-53cbdb455888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:29.069155 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5fae61-fb05-4da2-a7a0-53cbdb455888 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:29.141926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1312/2658] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:19:28 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:19:29.146257 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cca2e6-3369-4997-8e74-e4e981aa683c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:29.157548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cca2e6-3369-4997-8e74-e4e981aa683c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:29.162144 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d87c2-fb6e-4be0-a945-df40a33de49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:29.167388 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d87c2-fb6e-4be0-a945-df40a33de49f admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:29.167388 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d87c2-fb6e-4be0-a945-df40a33de49f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:29.167388 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d87c2-fb6e-4be0-a945-df40a33de49f admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:29.167388 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d87c2-fb6e-4be0-a945-df40a33de49f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:29.167388 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d87c2-fb6e-4be0-a945-df40a33de49f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:29.176483 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1347/2659] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:29.180276 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5fae61-fb05-4da2-a7a0-53cbdb455888 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:29.190061 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5fae61-fb05-4da2-a7a0-53cbdb455888 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:29.194606 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d20b4d-3dec-4d16-a41d-23ffed80edb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:29.195279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d20b4d-3dec-4d16-a41d-23ffed80edb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:29.195570 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d20b4d-3dec-4d16-a41d-23ffed80edb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:29.195841 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d20b4d-3dec-4d16-a41d-23ffed80edb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:29.196555 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d20b4d-3dec-4d16-a41d-23ffed80edb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:29.198641 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d20b4d-3dec-4d16-a41d-23ffed80edb0 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:29.208305 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1313/2660] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:29.212292 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d87c2-fb6e-4be0-a945-df40a33de49f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:29.229311 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d87c2-fb6e-4be0-a945-df40a33de49f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:29.233973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b1de8-35d3-4751-95b3-80469e846e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:29.234848 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b1de8-35d3-4751-95b3-80469e846e15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:29.235130 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b1de8-35d3-4751-95b3-80469e846e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:29.235458 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b1de8-35d3-4751-95b3-80469e846e15 admin admin] PATH_INFO: `/v3/projects/3782e858004e4317873c527df5eb405d/users/2766439e0c6e4c6a970652f386d8ae40/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:29.249647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b1de8-35d3-4751-95b3-80469e846e15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3782e858004e4317873c527df5eb405d, user_id=2766439e0c6e4c6a970652f386d8ae40, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:29.252256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b1de8-35d3-4751-95b3-80469e846e15 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:29.347443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1348/2661] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:19:29 2022] PUT /identity/v3/projects/3782e858004e4317873c527df5eb405d/users/2766439e0c6e4c6a970652f386d8ae40/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:29.358295 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d62da4-2616-4a58-8503-06f0ef5acffa None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:29.358616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d62da4-2616-4a58-8503-06f0ef5acffa None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:29.358898 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d62da4-2616-4a58-8503-06f0ef5acffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:29.385257 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c2d62da4-2616-4a58-8503-06f0ef5acffa None None] MFA Rules not processed for user `2766439e0c6e4c6a970652f386d8ae40`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:29.386457 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c2d62da4-2616-4a58-8503-06f0ef5acffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:29.427947 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:29.451105 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1314/2662] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:29.455629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b1de8-35d3-4751-95b3-80469e846e15 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:29.498400 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:29.519031 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b1de8-35d3-4751-95b3-80469e846e15 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:29.523862 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983a781-007a-49d7-b27f-112ecdf3be60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:29.524523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983a781-007a-49d7-b27f-112ecdf3be60 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:29.524819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983a781-007a-49d7-b27f-112ecdf3be60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:29.525103 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983a781-007a-49d7-b27f-112ecdf3be60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:29.525804 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9983a781-007a-49d7-b27f-112ecdf3be60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:29.528019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9983a781-007a-49d7-b27f-112ecdf3be60 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:29.623318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1349/2663] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:19:29 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:19:29.627440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d62da4-2616-4a58-8503-06f0ef5acffa None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:29.638705 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d62da4-2616-4a58-8503-06f0ef5acffa None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:29.692932 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede14b6-4d9b-4f20-8fbb-5b1a64fbce9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:29.693590 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede14b6-4d9b-4f20-8fbb-5b1a64fbce9c admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:29.693875 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede14b6-4d9b-4f20-8fbb-5b1a64fbce9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:29.694154 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede14b6-4d9b-4f20-8fbb-5b1a64fbce9c admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:29.694840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede14b6-4d9b-4f20-8fbb-5b1a64fbce9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:29.696785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede14b6-4d9b-4f20-8fbb-5b1a64fbce9c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:29.811212 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1315/2664] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:19:29 2022] POST /identity/v3/users => generated 296 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:19:29.815140 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983a781-007a-49d7-b27f-112ecdf3be60 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:29.825589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983a781-007a-49d7-b27f-112ecdf3be60 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:29.830206 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f724ba-8b44-41dc-b594-189cd86178f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:29.830870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f724ba-8b44-41dc-b594-189cd86178f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:29.831206 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f724ba-8b44-41dc-b594-189cd86178f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:29.831492 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f724ba-8b44-41dc-b594-189cd86178f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:29.832244 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f724ba-8b44-41dc-b594-189cd86178f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:29.833909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f724ba-8b44-41dc-b594-189cd86178f6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:29.845423 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1350/2665] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:29.849403 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede14b6-4d9b-4f20-8fbb-5b1a64fbce9c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:29.860156 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede14b6-4d9b-4f20-8fbb-5b1a64fbce9c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:29.864928 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f55fae-4fe9-41ec-864a-7537e84416fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:29.865794 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f55fae-4fe9-41ec-864a-7537e84416fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:29.866074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f55fae-4fe9-41ec-864a-7537e84416fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:29.866348 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f55fae-4fe9-41ec-864a-7537e84416fc admin admin] PATH_INFO: `/v3/projects/1577d95c60f844f6b2e35e12287dc91e/users/f9695248f0fe469483f96bdb671770e5/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:29.881758 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f55fae-4fe9-41ec-864a-7537e84416fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1577d95c60f844f6b2e35e12287dc91e, user_id=f9695248f0fe469483f96bdb671770e5, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:29.884562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f55fae-4fe9-41ec-864a-7537e84416fc admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:30.009056 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1316/2666] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:19:29 2022] PUT /identity/v3/projects/1577d95c60f844f6b2e35e12287dc91e/users/f9695248f0fe469483f96bdb671770e5/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:19:30.012983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f724ba-8b44-41dc-b594-189cd86178f6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:30.023456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f724ba-8b44-41dc-b594-189cd86178f6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:30.078405 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b64805-644f-4885-b7a0-132b9b9a258d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:30.079063 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b64805-644f-4885-b7a0-132b9b9a258d admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:30.079372 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b64805-644f-4885-b7a0-132b9b9a258d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:30.079649 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b64805-644f-4885-b7a0-132b9b9a258d admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:30.080363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b64805-644f-4885-b7a0-132b9b9a258d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:30.081957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b64805-644f-4885-b7a0-132b9b9a258d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:30.091506 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1351/2667] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:30.095565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f55fae-4fe9-41ec-864a-7537e84416fc admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:30.106074 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f55fae-4fe9-41ec-864a-7537e84416fc admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:30.110622 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857ebd3-c1a4-4b6e-b9e1-e8702b0b01d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:30.111518 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857ebd3-c1a4-4b6e-b9e1-e8702b0b01d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:30.111799 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857ebd3-c1a4-4b6e-b9e1-e8702b0b01d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:30.112073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857ebd3-c1a4-4b6e-b9e1-e8702b0b01d6 admin admin] PATH_INFO: `/v3/projects/1577d95c60f844f6b2e35e12287dc91e/users/f9695248f0fe469483f96bdb671770e5/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:30.114005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857ebd3-c1a4-4b6e-b9e1-e8702b0b01d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1577d95c60f844f6b2e35e12287dc91e, user_id=f9695248f0fe469483f96bdb671770e5, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:30.116664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857ebd3-c1a4-4b6e-b9e1-e8702b0b01d6 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:30.127150 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1317/2668] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:19:30 2022] PUT /identity/v3/projects/1577d95c60f844f6b2e35e12287dc91e/users/f9695248f0fe469483f96bdb671770e5/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:30.138603 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1096a9-00a0-41f7-87ca-6cf4116df2f9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:30.138902 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1096a9-00a0-41f7-87ca-6cf4116df2f9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:30.139212 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1096a9-00a0-41f7-87ca-6cf4116df2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:30.165711 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2f1096a9-00a0-41f7-87ca-6cf4116df2f9 None None] MFA Rules not processed for user `f9695248f0fe469483f96bdb671770e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:30.166929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2f1096a9-00a0-41f7-87ca-6cf4116df2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:30.233348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1352/2669] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:30.245558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857ebd3-c1a4-4b6e-b9e1-e8702b0b01d6 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:30.247269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9857ebd3-c1a4-4b6e-b9e1-e8702b0b01d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:30.304394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857ebd3-c1a4-4b6e-b9e1-e8702b0b01d6 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:30.308035 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d7b30-022d-4e74-a027-a060c62b5021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:30.308670 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d7b30-022d-4e74-a027-a060c62b5021 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:30.308941 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d7b30-022d-4e74-a027-a060c62b5021 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:30.309216 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d7b30-022d-4e74-a027-a060c62b5021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:30.318041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d7b30-022d-4e74-a027-a060c62b5021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:30.319505 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d7b30-022d-4e74-a027-a060c62b5021 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:30.328716 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1318/2670] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:19:30.429362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1096a9-00a0-41f7-87ca-6cf4116df2f9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:30.430757 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2f1096a9-00a0-41f7-87ca-6cf4116df2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:30.442610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1096a9-00a0-41f7-87ca-6cf4116df2f9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:30.446109 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cf8ac-aeab-4582-9ade-4638998dfa95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:30.446741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cf8ac-aeab-4582-9ade-4638998dfa95 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:30.447025 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cf8ac-aeab-4582-9ade-4638998dfa95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:30.447336 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cf8ac-aeab-4582-9ade-4638998dfa95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:30.477700 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:30.504713 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9cf8ac-aeab-4582-9ade-4638998dfa95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:30.506147 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9cf8ac-aeab-4582-9ade-4638998dfa95 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:30.515642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1353/2671] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:19:34.155750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d7b30-022d-4e74-a027-a060c62b5021 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:34.196656 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:34.217276 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d7b30-022d-4e74-a027-a060c62b5021 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:34.221864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed6082-7db4-4748-805c-d5213a26872d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:34.222770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed6082-7db4-4748-805c-d5213a26872d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:34.223298 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed6082-7db4-4748-805c-d5213a26872d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:34.223756 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed6082-7db4-4748-805c-d5213a26872d admin admin] PATH_INFO: `/v3/users/bba2c9a3a5ec46eea3e8e5467f34d92b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:34.225300 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed6082-7db4-4748-805c-d5213a26872d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bba2c9a3a5ec46eea3e8e5467f34d92b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:34.227263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed6082-7db4-4748-805c-d5213a26872d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:34.414888 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c5ed6082-7db4-4748-805c-d5213a26872d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bba2c9a3a5ec46eea3e8e5467f34d92b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:34.422583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c5ed6082-7db4-4748-805c-d5213a26872d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bba2c9a3a5ec46eea3e8e5467f34d92b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:34.423285 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c5ed6082-7db4-4748-805c-d5213a26872d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bba2c9a3a5ec46eea3e8e5467f34d92b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:34.427137 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c5ed6082-7db4-4748-805c-d5213a26872d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bba2c9a3a5ec46eea3e8e5467f34d92b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:34.436362 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c5ed6082-7db4-4748-805c-d5213a26872d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bba2c9a3a5ec46eea3e8e5467f34d92b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:34.519201 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1319/2672] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:19:34 2022] DELETE /identity/v3/users/bba2c9a3a5ec46eea3e8e5467f34d92b => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:34.523822 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d0f02-7291-4013-9249-13dfb9d475ff None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:34.537743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d0f02-7291-4013-9249-13dfb9d475ff None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:34.537743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d0f02-7291-4013-9249-13dfb9d475ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:34.541755 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-001d0f02-7291-4013-9249-13dfb9d475ff None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:34.543005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-001d0f02-7291-4013-9249-13dfb9d475ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:34.600657 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1354/2673] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:34.610297 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed6082-7db4-4748-805c-d5213a26872d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:34.611738 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c5ed6082-7db4-4748-805c-d5213a26872d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:34.669311 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed6082-7db4-4748-805c-d5213a26872d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:34.672945 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5cc82-842b-49da-84b1-1a6c0a20268d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:34.673589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5cc82-842b-49da-84b1-1a6c0a20268d service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:34.673879 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5cc82-842b-49da-84b1-1a6c0a20268d service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:34.674157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5cc82-842b-49da-84b1-1a6c0a20268d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:34.682961 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5cc82-842b-49da-84b1-1a6c0a20268d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:34.685047 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5cc82-842b-49da-84b1-1a6c0a20268d service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:34.694422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1320/2674] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:19:35.116026 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d0f02-7291-4013-9249-13dfb9d475ff None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:35.117379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-001d0f02-7291-4013-9249-13dfb9d475ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:35.136059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d0f02-7291-4013-9249-13dfb9d475ff None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:35.136059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870999b9-36af-4d54-9a32-3642fa7bb9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:35.136059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870999b9-36af-4d54-9a32-3642fa7bb9a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:35.136059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870999b9-36af-4d54-9a32-3642fa7bb9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:35.136375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870999b9-36af-4d54-9a32-3642fa7bb9a2 admin admin] PATH_INFO: `/v3/projects/c60eae7819eb4198adefa1df3616f2c3` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:35.136542 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870999b9-36af-4d54-9a32-3642fa7bb9a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c60eae7819eb4198adefa1df3616f2c3)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:35.138507 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870999b9-36af-4d54-9a32-3642fa7bb9a2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:35.254551 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-870999b9-36af-4d54-9a32-3642fa7bb9a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c60eae7819eb4198adefa1df3616f2c3', 'request_id': 'req-870999b9-36af-4d54-9a32-3642fa7bb9a2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:35.326155 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-870999b9-36af-4d54-9a32-3642fa7bb9a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c60eae7819eb4198adefa1df3616f2c3', 'request_id': 'req-870999b9-36af-4d54-9a32-3642fa7bb9a2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:35.332007 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-870999b9-36af-4d54-9a32-3642fa7bb9a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c60eae7819eb4198adefa1df3616f2c3', 'request_id': 'req-870999b9-36af-4d54-9a32-3642fa7bb9a2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:35.343039 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-870999b9-36af-4d54-9a32-3642fa7bb9a2 admin admin] The token cache is being invalidate because project c60eae7819eb4198adefa1df3616f2c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:19:35.343375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-870999b9-36af-4d54-9a32-3642fa7bb9a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:35.344039 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-870999b9-36af-4d54-9a32-3642fa7bb9a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:35.345587 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1355/2675] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:19:35 2022] DELETE /identity/v3/projects/c60eae7819eb4198adefa1df3616f2c3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:35.349568 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5cc82-842b-49da-84b1-1a6c0a20268d service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:35.359463 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5cc82-842b-49da-84b1-1a6c0a20268d service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:35.415034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c2a02-b138-4b59-ab20-2f62baabc946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:35.415879 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c2a02-b138-4b59-ab20-2f62baabc946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:35.416169 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c2a02-b138-4b59-ab20-2f62baabc946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:35.416448 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c2a02-b138-4b59-ab20-2f62baabc946 admin admin] PATH_INFO: `/v3/users/2766439e0c6e4c6a970652f386d8ae40` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:35.418165 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c2a02-b138-4b59-ab20-2f62baabc946 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2766439e0c6e4c6a970652f386d8ae40)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:35.420150 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c2a02-b138-4b59-ab20-2f62baabc946 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:35.421626 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:35.596876 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f34c2a02-b138-4b59-ab20-2f62baabc946 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2766439e0c6e4c6a970652f386d8ae40'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:35.604007 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f34c2a02-b138-4b59-ab20-2f62baabc946 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2766439e0c6e4c6a970652f386d8ae40'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:35.604692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f34c2a02-b138-4b59-ab20-2f62baabc946 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2766439e0c6e4c6a970652f386d8ae40'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:35.605854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f34c2a02-b138-4b59-ab20-2f62baabc946 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2766439e0c6e4c6a970652f386d8ae40'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:35.617470 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f34c2a02-b138-4b59-ab20-2f62baabc946 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2766439e0c6e4c6a970652f386d8ae40'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:35.686086 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1321/2676] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:19:35 2022] DELETE /identity/v3/users/2766439e0c6e4c6a970652f386d8ae40 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:36.113042 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870999b9-36af-4d54-9a32-3642fa7bb9a2 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:36.114768 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-870999b9-36af-4d54-9a32-3642fa7bb9a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:36.128041 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870999b9-36af-4d54-9a32-3642fa7bb9a2 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:36.190628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:36.197534 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76046dc8-d7e8-420c-a949-41ebbf636a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:36.198403 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76046dc8-d7e8-420c-a949-41ebbf636a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:36.199272 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76046dc8-d7e8-420c-a949-41ebbf636a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:36.199551 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76046dc8-d7e8-420c-a949-41ebbf636a74 admin admin] PATH_INFO: `/v3/projects/3782e858004e4317873c527df5eb405d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:36.200720 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76046dc8-d7e8-420c-a949-41ebbf636a74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3782e858004e4317873c527df5eb405d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:36.202619 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76046dc8-d7e8-420c-a949-41ebbf636a74 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:36.313032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-76046dc8-d7e8-420c-a949-41ebbf636a74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3782e858004e4317873c527df5eb405d', 'request_id': 'req-76046dc8-d7e8-420c-a949-41ebbf636a74'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:36.394733 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-76046dc8-d7e8-420c-a949-41ebbf636a74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3782e858004e4317873c527df5eb405d', 'request_id': 'req-76046dc8-d7e8-420c-a949-41ebbf636a74'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:36.400604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-76046dc8-d7e8-420c-a949-41ebbf636a74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3782e858004e4317873c527df5eb405d', 'request_id': 'req-76046dc8-d7e8-420c-a949-41ebbf636a74'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:36.411907 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-76046dc8-d7e8-420c-a949-41ebbf636a74 admin admin] The token cache is being invalidate because project 3782e858004e4317873c527df5eb405d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:19:36.412191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-76046dc8-d7e8-420c-a949-41ebbf636a74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:36.412843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-76046dc8-d7e8-420c-a949-41ebbf636a74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:36.414384 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1356/2677] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:19:36 2022] DELETE /identity/v3/projects/3782e858004e4317873c527df5eb405d => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:36.418996 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c2a02-b138-4b59-ab20-2f62baabc946 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:36.420439 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f34c2a02-b138-4b59-ab20-2f62baabc946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:36.433831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c2a02-b138-4b59-ab20-2f62baabc946 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:36.488990 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d59b11-c409-42c8-900c-c5cd5f9be919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:36.489799 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d59b11-c409-42c8-900c-c5cd5f9be919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:36.490087 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d59b11-c409-42c8-900c-c5cd5f9be919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:36.490366 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d59b11-c409-42c8-900c-c5cd5f9be919 admin admin] PATH_INFO: `/v3/users/f9695248f0fe469483f96bdb671770e5` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:36.491617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d59b11-c409-42c8-900c-c5cd5f9be919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9695248f0fe469483f96bdb671770e5)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:36.493553 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d59b11-c409-42c8-900c-c5cd5f9be919 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:36.648241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f0d59b11-c409-42c8-900c-c5cd5f9be919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9695248f0fe469483f96bdb671770e5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:36.655442 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f0d59b11-c409-42c8-900c-c5cd5f9be919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9695248f0fe469483f96bdb671770e5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:36.656098 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f0d59b11-c409-42c8-900c-c5cd5f9be919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9695248f0fe469483f96bdb671770e5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:36.656800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f0d59b11-c409-42c8-900c-c5cd5f9be919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9695248f0fe469483f96bdb671770e5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:36.668523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f0d59b11-c409-42c8-900c-c5cd5f9be919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9695248f0fe469483f96bdb671770e5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:36.759732 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:36.765704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1322/2678] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:19:36 2022] DELETE /identity/v3/users/f9695248f0fe469483f96bdb671770e5 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:19:37.145102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76046dc8-d7e8-420c-a949-41ebbf636a74 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:37.146605 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-76046dc8-d7e8-420c-a949-41ebbf636a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:37.160422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76046dc8-d7e8-420c-a949-41ebbf636a74 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:37.217278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a47444-6089-43f6-b57f-2b1d9ac1901a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:37.218152 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a47444-6089-43f6-b57f-2b1d9ac1901a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:37.218440 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a47444-6089-43f6-b57f-2b1d9ac1901a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:37.218719 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a47444-6089-43f6-b57f-2b1d9ac1901a admin admin] PATH_INFO: `/v3/projects/1577d95c60f844f6b2e35e12287dc91e` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:37.219927 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a47444-6089-43f6-b57f-2b1d9ac1901a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1577d95c60f844f6b2e35e12287dc91e)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:37.221857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a47444-6089-43f6-b57f-2b1d9ac1901a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:37.333462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-61a47444-6089-43f6-b57f-2b1d9ac1901a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1577d95c60f844f6b2e35e12287dc91e', 'request_id': 'req-61a47444-6089-43f6-b57f-2b1d9ac1901a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:37.400561 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-61a47444-6089-43f6-b57f-2b1d9ac1901a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1577d95c60f844f6b2e35e12287dc91e', 'request_id': 'req-61a47444-6089-43f6-b57f-2b1d9ac1901a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:37.407279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-61a47444-6089-43f6-b57f-2b1d9ac1901a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1577d95c60f844f6b2e35e12287dc91e', 'request_id': 'req-61a47444-6089-43f6-b57f-2b1d9ac1901a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:37.418671 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-61a47444-6089-43f6-b57f-2b1d9ac1901a admin admin] The token cache is being invalidate because project 1577d95c60f844f6b2e35e12287dc91e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:19:37.419023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-61a47444-6089-43f6-b57f-2b1d9ac1901a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:37.421334 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-61a47444-6089-43f6-b57f-2b1d9ac1901a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:37.421709 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1357/2679] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:19:37 2022] DELETE /identity/v3/projects/1577d95c60f844f6b2e35e12287dc91e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:37.427559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f62c5d-daa4-4259-872a-b3ff5dede552 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:37.428039 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f62c5d-daa4-4259-872a-b3ff5dede552 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:37.428139 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f62c5d-daa4-4259-872a-b3ff5dede552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:37.445717 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c9f62c5d-daa4-4259-872a-b3ff5dede552 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:37.446899 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c9f62c5d-daa4-4259-872a-b3ff5dede552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:37.506606 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1323/2680] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:37.515530 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06fec2-7ef7-4db9-8288-77d205d867ec None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:37.515830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06fec2-7ef7-4db9-8288-77d205d867ec None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:37.516114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06fec2-7ef7-4db9-8288-77d205d867ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:37.535408 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2b06fec2-7ef7-4db9-8288-77d205d867ec None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:37.536683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2b06fec2-7ef7-4db9-8288-77d205d867ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:37.544531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1358/2681] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:37.548688 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f62c5d-daa4-4259-872a-b3ff5dede552 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:37.558833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f62c5d-daa4-4259-872a-b3ff5dede552 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:37.563411 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65d08d-2e2b-41f7-ab54-ac81b1a78822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:37.564057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65d08d-2e2b-41f7-ab54-ac81b1a78822 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:37.564343 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65d08d-2e2b-41f7-ab54-ac81b1a78822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:37.564618 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65d08d-2e2b-41f7-ab54-ac81b1a78822 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:37.565338 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65d08d-2e2b-41f7-ab54-ac81b1a78822 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:19:37.565647 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65d08d-2e2b-41f7-ab54-ac81b1a78822 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:37.567264 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65d08d-2e2b-41f7-ab54-ac81b1a78822 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:37.579099 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1324/2682] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 19:19:37.583131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06fec2-7ef7-4db9-8288-77d205d867ec None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:37.592631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06fec2-7ef7-4db9-8288-77d205d867ec None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:37.597323 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994597f1-3037-478e-b755-bb8f09a19b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:37.597969 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994597f1-3037-478e-b755-bb8f09a19b00 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:37.598259 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994597f1-3037-478e-b755-bb8f09a19b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:37.598539 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994597f1-3037-478e-b755-bb8f09a19b00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:37.599263 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994597f1-3037-478e-b755-bb8f09a19b00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:37.601159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994597f1-3037-478e-b755-bb8f09a19b00 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:37.605163 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:37.699528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1359/2683] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:19:37 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:19:37.703622 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65d08d-2e2b-41f7-ab54-ac81b1a78822 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:37.714160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65d08d-2e2b-41f7-ab54-ac81b1a78822 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:37.772959 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc44f54-c073-4514-983c-faad47141e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:37.773645 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc44f54-c073-4514-983c-faad47141e3c admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:37.773935 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc44f54-c073-4514-983c-faad47141e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:37.774226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc44f54-c073-4514-983c-faad47141e3c admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:37.774916 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc44f54-c073-4514-983c-faad47141e3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:37.776901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc44f54-c073-4514-983c-faad47141e3c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:37.857301 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1325/2684] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:19:37 2022] POST /identity/v3/users => generated 291 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:19:37.861425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994597f1-3037-478e-b755-bb8f09a19b00 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:37.872464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994597f1-3037-478e-b755-bb8f09a19b00 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:37.877102 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdd9ae-7363-4e89-a4ac-e160667143d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:37.879407 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdd9ae-7363-4e89-a4ac-e160667143d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:37.879407 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdd9ae-7363-4e89-a4ac-e160667143d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:37.879407 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdd9ae-7363-4e89-a4ac-e160667143d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:37.879407 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bdd9ae-7363-4e89-a4ac-e160667143d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:37.880676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bdd9ae-7363-4e89-a4ac-e160667143d5 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:37.891849 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1360/2685] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:37.895800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc44f54-c073-4514-983c-faad47141e3c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:37.906668 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc44f54-c073-4514-983c-faad47141e3c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:37.911329 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43024a3-eed4-470a-bf8e-6cafec7a6b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:37.911998 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43024a3-eed4-470a-bf8e-6cafec7a6b10 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:37.912282 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43024a3-eed4-470a-bf8e-6cafec7a6b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:37.912559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43024a3-eed4-470a-bf8e-6cafec7a6b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:37.913283 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43024a3-eed4-470a-bf8e-6cafec7a6b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:37.914873 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43024a3-eed4-470a-bf8e-6cafec7a6b10 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:37.927283 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1326/2686] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:19:37.931673 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdd9ae-7363-4e89-a4ac-e160667143d5 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:37.942530 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdd9ae-7363-4e89-a4ac-e160667143d5 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:37.947390 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b68ca-4f04-42ce-9771-c5adb245d7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:37.948283 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b68ca-4f04-42ce-9771-c5adb245d7ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:37.948599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b68ca-4f04-42ce-9771-c5adb245d7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:37.948883 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b68ca-4f04-42ce-9771-c5adb245d7ea admin admin] PATH_INFO: `/v3/projects/cc60cfe94155494fa1402b4fb7b0f295/users/f70be843b5f045649d150a2d816b713a/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:37.964915 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b68ca-4f04-42ce-9771-c5adb245d7ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc60cfe94155494fa1402b4fb7b0f295, user_id=f70be843b5f045649d150a2d816b713a, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:37.967613 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b68ca-4f04-42ce-9771-c5adb245d7ea admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:38.099687 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1361/2687] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:19:37 2022] PUT /identity/v3/projects/cc60cfe94155494fa1402b4fb7b0f295/users/f70be843b5f045649d150a2d816b713a/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:19:38.111594 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ec57b-4ba5-4ada-8924-ed23ecd41b74 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:38.111813 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ec57b-4ba5-4ada-8924-ed23ecd41b74 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:38.112097 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ec57b-4ba5-4ada-8924-ed23ecd41b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:38.139433 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-6f6ec57b-4ba5-4ada-8924-ed23ecd41b74 None None] MFA Rules not processed for user `f70be843b5f045649d150a2d816b713a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:38.140657 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6f6ec57b-4ba5-4ada-8924-ed23ecd41b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:38.196887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:38.206828 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1327/2688] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:38.211314 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b68ca-4f04-42ce-9771-c5adb245d7ea admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:38.286160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b68ca-4f04-42ce-9771-c5adb245d7ea admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:38.290852 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f373177-8bd8-4b6a-8db5-06ed32fd2d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:38.291533 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f373177-8bd8-4b6a-8db5-06ed32fd2d69 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:38.291822 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f373177-8bd8-4b6a-8db5-06ed32fd2d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:38.292099 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f373177-8bd8-4b6a-8db5-06ed32fd2d69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:38.293448 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f373177-8bd8-4b6a-8db5-06ed32fd2d69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:38.295381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f373177-8bd8-4b6a-8db5-06ed32fd2d69 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:38.362965 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1362/2689] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:19:38 2022] POST /identity/v3/projects => generated 381 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:19:38.368617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ec57b-4ba5-4ada-8924-ed23ecd41b74 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:38.381374 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ec57b-4ba5-4ada-8924-ed23ecd41b74 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:38.446572 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1eedfa-8db1-41c7-9b7a-f57a2379b66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:38.447290 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1eedfa-8db1-41c7-9b7a-f57a2379b66f admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:38.447290 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1eedfa-8db1-41c7-9b7a-f57a2379b66f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:38.447514 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1eedfa-8db1-41c7-9b7a-f57a2379b66f admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:38.448057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1eedfa-8db1-41c7-9b7a-f57a2379b66f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:38.450661 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1eedfa-8db1-41c7-9b7a-f57a2379b66f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:38.537974 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1328/2690] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:19:38 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:19:38.539362 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f373177-8bd8-4b6a-8db5-06ed32fd2d69 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:38.554991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f373177-8bd8-4b6a-8db5-06ed32fd2d69 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:38.561582 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb741a55-d741-49ba-993f-4e498a8dbf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:38.562413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb741a55-d741-49ba-993f-4e498a8dbf69 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:38.562702 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb741a55-d741-49ba-993f-4e498a8dbf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:38.562992 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb741a55-d741-49ba-993f-4e498a8dbf69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:38.563773 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb741a55-d741-49ba-993f-4e498a8dbf69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:38.565450 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb741a55-d741-49ba-993f-4e498a8dbf69 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:38.582618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1363/2691] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:38.584445 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1eedfa-8db1-41c7-9b7a-f57a2379b66f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:38.596472 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1eedfa-8db1-41c7-9b7a-f57a2379b66f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:38.604017 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46324c-ce6e-4e47-8bf2-c0fb7be956bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:38.605529 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46324c-ce6e-4e47-8bf2-c0fb7be956bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:38.605822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46324c-ce6e-4e47-8bf2-c0fb7be956bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:38.606108 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46324c-ce6e-4e47-8bf2-c0fb7be956bf admin admin] PATH_INFO: `/v3/projects/a4f91f67c86447daa76a20f591bbf5b1/users/0cc21f7efccc42d0ae5823fca2942dd4/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:38.627598 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46324c-ce6e-4e47-8bf2-c0fb7be956bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f91f67c86447daa76a20f591bbf5b1, user_id=0cc21f7efccc42d0ae5823fca2942dd4, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:38.630355 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46324c-ce6e-4e47-8bf2-c0fb7be956bf admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:38.689417 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1329/2692] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:19:38 2022] PUT /identity/v3/projects/a4f91f67c86447daa76a20f591bbf5b1/users/0cc21f7efccc42d0ae5823fca2942dd4/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:38.693752 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb741a55-d741-49ba-993f-4e498a8dbf69 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:38.717687 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb741a55-d741-49ba-993f-4e498a8dbf69 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:38.799371 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4736b-abc3-4dd2-b744-4abcdba8cbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:38.800268 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4736b-abc3-4dd2-b744-4abcdba8cbf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:38.800628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4736b-abc3-4dd2-b744-4abcdba8cbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:38.800905 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4736b-abc3-4dd2-b744-4abcdba8cbf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:38.801707 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4736b-abc3-4dd2-b744-4abcdba8cbf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:38.803538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4736b-abc3-4dd2-b744-4abcdba8cbf8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:38.814301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1364/2693] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:38.818489 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46324c-ce6e-4e47-8bf2-c0fb7be956bf admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:38.830153 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46324c-ce6e-4e47-8bf2-c0fb7be956bf admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:38.840664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c33376-00bf-4f43-81b5-938612da774c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:38.841864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c33376-00bf-4f43-81b5-938612da774c admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:38.842053 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c33376-00bf-4f43-81b5-938612da774c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:38.842652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c33376-00bf-4f43-81b5-938612da774c admin admin] PATH_INFO: `/v3/projects/a4f91f67c86447daa76a20f591bbf5b1/users/0cc21f7efccc42d0ae5823fca2942dd4/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:38.844707 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c33376-00bf-4f43-81b5-938612da774c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f91f67c86447daa76a20f591bbf5b1, user_id=0cc21f7efccc42d0ae5823fca2942dd4, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:38.847488 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c33376-00bf-4f43-81b5-938612da774c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:38.860729 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1330/2694] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:19:38 2022] PUT /identity/v3/projects/a4f91f67c86447daa76a20f591bbf5b1/users/0cc21f7efccc42d0ae5823fca2942dd4/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:19:38.932900 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c0524-49b6-4372-8bd9-8b7c7d8032ac None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:38.933836 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c0524-49b6-4372-8bd9-8b7c7d8032ac None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:38.934120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c0524-49b6-4372-8bd9-8b7c7d8032ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:38.962599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-271c0524-49b6-4372-8bd9-8b7c7d8032ac None None] MFA Rules not processed for user `0cc21f7efccc42d0ae5823fca2942dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:38.963870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-271c0524-49b6-4372-8bd9-8b7c7d8032ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:38.974386 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:39.041189 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1365/2695] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:39.052367 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c33376-00bf-4f43-81b5-938612da774c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:39.054043 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-77c33376-00bf-4f43-81b5-938612da774c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:39.112229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c33376-00bf-4f43-81b5-938612da774c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:39.115929 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c5f86-af9f-40a8-9178-759dde5f17dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:39.116589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c5f86-af9f-40a8-9178-759dde5f17dd service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:39.116875 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c5f86-af9f-40a8-9178-759dde5f17dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:39.117157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c5f86-af9f-40a8-9178-759dde5f17dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:39.126564 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c5f86-af9f-40a8-9178-759dde5f17dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:39.128076 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c5f86-af9f-40a8-9178-759dde5f17dd service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:39.139566 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1331/2696] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:19:39.695571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c0524-49b6-4372-8bd9-8b7c7d8032ac None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:39.770246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c0524-49b6-4372-8bd9-8b7c7d8032ac None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:39.774930 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66735120-59eb-43fd-a2f0-aef0923b029d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:39.775808 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66735120-59eb-43fd-a2f0-aef0923b029d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:39.776095 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66735120-59eb-43fd-a2f0-aef0923b029d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:39.776374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66735120-59eb-43fd-a2f0-aef0923b029d admin admin] PATH_INFO: `/v3/users/f70be843b5f045649d150a2d816b713a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:39.777830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66735120-59eb-43fd-a2f0-aef0923b029d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f70be843b5f045649d150a2d816b713a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:39.779787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66735120-59eb-43fd-a2f0-aef0923b029d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:39.941487 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-66735120-59eb-43fd-a2f0-aef0923b029d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f70be843b5f045649d150a2d816b713a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:39.942078 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-66735120-59eb-43fd-a2f0-aef0923b029d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f70be843b5f045649d150a2d816b713a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:39.954411 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-66735120-59eb-43fd-a2f0-aef0923b029d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f70be843b5f045649d150a2d816b713a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:39.955105 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-66735120-59eb-43fd-a2f0-aef0923b029d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f70be843b5f045649d150a2d816b713a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:39.962755 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-66735120-59eb-43fd-a2f0-aef0923b029d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f70be843b5f045649d150a2d816b713a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:40.030518 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1366/2697] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:19:39 2022] DELETE /identity/v3/users/f70be843b5f045649d150a2d816b713a => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:40.035312 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649840a-7750-4b29-bed6-fee602fddfb6 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:40.035785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649840a-7750-4b29-bed6-fee602fddfb6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:40.036001 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649840a-7750-4b29-bed6-fee602fddfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:40.054283 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4649840a-7750-4b29-bed6-fee602fddfb6 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:40.055523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4649840a-7750-4b29-bed6-fee602fddfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:40.100614 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:40.116120 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1332/2698] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:40.127135 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66735120-59eb-43fd-a2f0-aef0923b029d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:40.128748 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-66735120-59eb-43fd-a2f0-aef0923b029d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:40.162254 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:40.188406 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66735120-59eb-43fd-a2f0-aef0923b029d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:40.192223 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b7f7d-79d2-45bc-a5c0-fa5719ef27c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:40.192810 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b7f7d-79d2-45bc-a5c0-fa5719ef27c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:40.193107 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b7f7d-79d2-45bc-a5c0-fa5719ef27c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:40.193387 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b7f7d-79d2-45bc-a5c0-fa5719ef27c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:40.202838 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b7f7d-79d2-45bc-a5c0-fa5719ef27c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:40.204348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b7f7d-79d2-45bc-a5c0-fa5719ef27c8 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:40.213705 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1367/2699] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:19:40.675311 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649840a-7750-4b29-bed6-fee602fddfb6 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:40.676889 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4649840a-7750-4b29-bed6-fee602fddfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:40.689565 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649840a-7750-4b29-bed6-fee602fddfb6 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:40.694285 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38565ba0-034d-465e-b0af-5f80e3129aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:40.695212 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38565ba0-034d-465e-b0af-5f80e3129aab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:40.695507 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38565ba0-034d-465e-b0af-5f80e3129aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:40.695786 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38565ba0-034d-465e-b0af-5f80e3129aab admin admin] PATH_INFO: `/v3/projects/cc60cfe94155494fa1402b4fb7b0f295` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:40.697127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38565ba0-034d-465e-b0af-5f80e3129aab admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc60cfe94155494fa1402b4fb7b0f295)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:40.699043 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38565ba0-034d-465e-b0af-5f80e3129aab admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:40.822383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-38565ba0-034d-465e-b0af-5f80e3129aab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc60cfe94155494fa1402b4fb7b0f295', 'request_id': 'req-38565ba0-034d-465e-b0af-5f80e3129aab'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:40.886919 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-38565ba0-034d-465e-b0af-5f80e3129aab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc60cfe94155494fa1402b4fb7b0f295', 'request_id': 'req-38565ba0-034d-465e-b0af-5f80e3129aab'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:40.892946 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-38565ba0-034d-465e-b0af-5f80e3129aab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc60cfe94155494fa1402b4fb7b0f295', 'request_id': 'req-38565ba0-034d-465e-b0af-5f80e3129aab'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:40.905124 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-38565ba0-034d-465e-b0af-5f80e3129aab admin admin] The token cache is being invalidate because project cc60cfe94155494fa1402b4fb7b0f295 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:19:40.905415 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-38565ba0-034d-465e-b0af-5f80e3129aab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:40.906052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-38565ba0-034d-465e-b0af-5f80e3129aab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:40.907635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1333/2700] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:19:40 2022] DELETE /identity/v3/projects/cc60cfe94155494fa1402b4fb7b0f295 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:40.911623 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b7f7d-79d2-45bc-a5c0-fa5719ef27c8 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:40.921577 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b7f7d-79d2-45bc-a5c0-fa5719ef27c8 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:40.982481 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb921c2-f452-4f6f-b1ca-70d067c284a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:40.983397 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb921c2-f452-4f6f-b1ca-70d067c284a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:40.983693 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb921c2-f452-4f6f-b1ca-70d067c284a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:40.983976 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb921c2-f452-4f6f-b1ca-70d067c284a0 admin admin] PATH_INFO: `/v3/users/0cc21f7efccc42d0ae5823fca2942dd4` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:40.985170 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb921c2-f452-4f6f-b1ca-70d067c284a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cc21f7efccc42d0ae5823fca2942dd4)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:40.987121 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb921c2-f452-4f6f-b1ca-70d067c284a0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:41.186646 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3eb921c2-f452-4f6f-b1ca-70d067c284a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cc21f7efccc42d0ae5823fca2942dd4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:41.187336 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3eb921c2-f452-4f6f-b1ca-70d067c284a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cc21f7efccc42d0ae5823fca2942dd4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:41.199713 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3eb921c2-f452-4f6f-b1ca-70d067c284a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cc21f7efccc42d0ae5823fca2942dd4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:41.200357 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3eb921c2-f452-4f6f-b1ca-70d067c284a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cc21f7efccc42d0ae5823fca2942dd4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:41.207126 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3eb921c2-f452-4f6f-b1ca-70d067c284a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cc21f7efccc42d0ae5823fca2942dd4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:41.292884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1368/2701] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:19:40 2022] DELETE /identity/v3/users/0cc21f7efccc42d0ae5823fca2942dd4 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:41.692500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38565ba0-034d-465e-b0af-5f80e3129aab admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:41.694672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-38565ba0-034d-465e-b0af-5f80e3129aab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:41.709836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38565ba0-034d-465e-b0af-5f80e3129aab admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:41.767701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe3b31-9317-4bfc-8e25-8879bae722c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:41.768571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe3b31-9317-4bfc-8e25-8879bae722c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:41.768858 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe3b31-9317-4bfc-8e25-8879bae722c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:41.769134 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe3b31-9317-4bfc-8e25-8879bae722c9 admin admin] PATH_INFO: `/v3/projects/a4f91f67c86447daa76a20f591bbf5b1` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:41.770313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe3b31-9317-4bfc-8e25-8879bae722c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4f91f67c86447daa76a20f591bbf5b1)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:41.772256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe3b31-9317-4bfc-8e25-8879bae722c9 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:41.905168 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8dbe3b31-9317-4bfc-8e25-8879bae722c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4f91f67c86447daa76a20f591bbf5b1', 'request_id': 'req-8dbe3b31-9317-4bfc-8e25-8879bae722c9'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:42.012525 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8dbe3b31-9317-4bfc-8e25-8879bae722c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4f91f67c86447daa76a20f591bbf5b1', 'request_id': 'req-8dbe3b31-9317-4bfc-8e25-8879bae722c9'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:42.018494 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8dbe3b31-9317-4bfc-8e25-8879bae722c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4f91f67c86447daa76a20f591bbf5b1', 'request_id': 'req-8dbe3b31-9317-4bfc-8e25-8879bae722c9'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:42.031599 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8dbe3b31-9317-4bfc-8e25-8879bae722c9 admin admin] The token cache is being invalidate because project a4f91f67c86447daa76a20f591bbf5b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:19:42.031896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8dbe3b31-9317-4bfc-8e25-8879bae722c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:42.032534 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8dbe3b31-9317-4bfc-8e25-8879bae722c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:42.034062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1334/2702] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:19:41 2022] DELETE /identity/v3/projects/a4f91f67c86447daa76a20f591bbf5b1 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:42.039087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5adb1c-6078-4d65-8086-e607e92a37d5 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:42.039583 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5adb1c-6078-4d65-8086-e607e92a37d5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:42.039736 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5adb1c-6078-4d65-8086-e607e92a37d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:42.058285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9e5adb1c-6078-4d65-8086-e607e92a37d5 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:42.059620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9e5adb1c-6078-4d65-8086-e607e92a37d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:42.226723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:42.242107 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1369/2703] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:19:42.250776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906669c-8502-41ce-aa21-d30fa9584491 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:42.251009 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906669c-8502-41ce-aa21-d30fa9584491 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:42.251339 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906669c-8502-41ce-aa21-d30fa9584491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:42.257937 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:42.270378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-5906669c-8502-41ce-aa21-d30fa9584491 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:42.271725 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5906669c-8502-41ce-aa21-d30fa9584491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:42.279932 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1335/2704] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:42.284117 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5adb1c-6078-4d65-8086-e607e92a37d5 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:42.294125 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5adb1c-6078-4d65-8086-e607e92a37d5 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:42.298819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49130747-50c4-45c3-ba93-3bf437a709d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:42.299516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49130747-50c4-45c3-ba93-3bf437a709d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:42.299809 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49130747-50c4-45c3-ba93-3bf437a709d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:42.300092 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49130747-50c4-45c3-ba93-3bf437a709d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:42.300897 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49130747-50c4-45c3-ba93-3bf437a709d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:19:42.301209 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49130747-50c4-45c3-ba93-3bf437a709d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:42.302807 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49130747-50c4-45c3-ba93-3bf437a709d6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:42.316293 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1370/2705] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:19:42.319431 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906669c-8502-41ce-aa21-d30fa9584491 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:42.329086 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906669c-8502-41ce-aa21-d30fa9584491 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:42.334169 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be1a54-7730-4844-8f5a-8468e4bbb925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:42.334855 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be1a54-7730-4844-8f5a-8468e4bbb925 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:42.335139 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be1a54-7730-4844-8f5a-8468e4bbb925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:42.339780 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be1a54-7730-4844-8f5a-8468e4bbb925 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:42.339780 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be1a54-7730-4844-8f5a-8468e4bbb925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:42.339780 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be1a54-7730-4844-8f5a-8468e4bbb925 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:42.434924 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1336/2706] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:19:42 2022] POST /identity/v3/projects => generated 363 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:19:42.438989 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49130747-50c4-45c3-ba93-3bf437a709d6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:42.448769 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49130747-50c4-45c3-ba93-3bf437a709d6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:42.504844 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58718395-3214-4f26-9387-907720e967ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:42.505435 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58718395-3214-4f26-9387-907720e967ec admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:42.505719 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58718395-3214-4f26-9387-907720e967ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:42.505997 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58718395-3214-4f26-9387-907720e967ec admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:42.506702 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58718395-3214-4f26-9387-907720e967ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:42.508641 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58718395-3214-4f26-9387-907720e967ec admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:42.588719 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1371/2707] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:19:42 2022] POST /identity/v3/users => generated 282 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:19:42.593335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be1a54-7730-4844-8f5a-8468e4bbb925 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:42.603480 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be1a54-7730-4844-8f5a-8468e4bbb925 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:42.610130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57d14c-6dc8-4595-aa9d-b36a3c22acc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:42.610799 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57d14c-6dc8-4595-aa9d-b36a3c22acc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:42.611107 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57d14c-6dc8-4595-aa9d-b36a3c22acc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:42.611402 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57d14c-6dc8-4595-aa9d-b36a3c22acc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:42.612133 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac57d14c-6dc8-4595-aa9d-b36a3c22acc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:42.613733 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac57d14c-6dc8-4595-aa9d-b36a3c22acc9 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:42.624267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1337/2708] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:42.628218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58718395-3214-4f26-9387-907720e967ec admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:42.638462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58718395-3214-4f26-9387-907720e967ec admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:42.643219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d824c08-cf89-4075-9403-c431053fa2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:42.643881 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d824c08-cf89-4075-9403-c431053fa2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:42.644172 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d824c08-cf89-4075-9403-c431053fa2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:42.644456 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d824c08-cf89-4075-9403-c431053fa2c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:42.645186 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d824c08-cf89-4075-9403-c431053fa2c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:42.646790 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d824c08-cf89-4075-9403-c431053fa2c2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:42.657481 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1372/2709] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:42.661639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57d14c-6dc8-4595-aa9d-b36a3c22acc9 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:42.672328 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57d14c-6dc8-4595-aa9d-b36a3c22acc9 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:42.677007 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597a258-053f-4f7c-8154-deeb4445b252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:42.677874 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597a258-053f-4f7c-8154-deeb4445b252 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:42.678158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597a258-053f-4f7c-8154-deeb4445b252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:42.678435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597a258-053f-4f7c-8154-deeb4445b252 admin admin] PATH_INFO: `/v3/projects/8e5c89bd4ec644daa5ffff4767c540ec/users/c65f75c2f1f54ec28dcc4bb3fd3b818a/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:42.693127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597a258-053f-4f7c-8154-deeb4445b252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e5c89bd4ec644daa5ffff4767c540ec, user_id=c65f75c2f1f54ec28dcc4bb3fd3b818a, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:42.695922 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597a258-053f-4f7c-8154-deeb4445b252 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:42.772096 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1338/2710] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:19:42 2022] PUT /identity/v3/projects/8e5c89bd4ec644daa5ffff4767c540ec/users/c65f75c2f1f54ec28dcc4bb3fd3b818a/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:19:42.782857 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad2f9d-b7fd-4316-978d-bc977eaa48c6 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:42.783146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad2f9d-b7fd-4316-978d-bc977eaa48c6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:42.783462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad2f9d-b7fd-4316-978d-bc977eaa48c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:42.809648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9aad2f9d-b7fd-4316-978d-bc977eaa48c6 None None] MFA Rules not processed for user `c65f75c2f1f54ec28dcc4bb3fd3b818a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:42.810886 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9aad2f9d-b7fd-4316-978d-bc977eaa48c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:42.873260 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1373/2711] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:42 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:42.884892 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597a258-053f-4f7c-8154-deeb4445b252 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:42.887398 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d597a258-053f-4f7c-8154-deeb4445b252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:42.944200 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597a258-053f-4f7c-8154-deeb4445b252 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:42.947839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087133e-7243-4a09-ad89-8406e25455d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:42.948508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087133e-7243-4a09-ad89-8406e25455d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:42.948822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087133e-7243-4a09-ad89-8406e25455d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:42.949102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087133e-7243-4a09-ad89-8406e25455d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:42.958982 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087133e-7243-4a09-ad89-8406e25455d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:42.960455 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087133e-7243-4a09-ad89-8406e25455d4 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:42.969612 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1339/2712] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:42 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:19:43.055803 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad2f9d-b7fd-4316-978d-bc977eaa48c6 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:43.057180 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9aad2f9d-b7fd-4316-978d-bc977eaa48c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:43.104466 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:43.115276 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad2f9d-b7fd-4316-978d-bc977eaa48c6 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:43.118917 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad9656-dfe4-48a0-ad8c-ab311cb6c19d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:43.119601 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad9656-dfe4-48a0-ad8c-ab311cb6c19d service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:43.119898 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad9656-dfe4-48a0-ad8c-ab311cb6c19d service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:43.120176 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad9656-dfe4-48a0-ad8c-ab311cb6c19d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:43.121924 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c2ad9656-dfe4-48a0-ad8c-ab311cb6c19d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:43.183305 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad9656-dfe4-48a0-ad8c-ab311cb6c19d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:43.184744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad9656-dfe4-48a0-ad8c-ab311cb6c19d service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:43.194489 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1374/2713] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:19:46.859258 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087133e-7243-4a09-ad89-8406e25455d4 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:46.860719 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7087133e-7243-4a09-ad89-8406e25455d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:46.875997 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087133e-7243-4a09-ad89-8406e25455d4 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:46.880659 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87690fa-7666-4941-88a9-4f6e8aa48cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:46.881564 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87690fa-7666-4941-88a9-4f6e8aa48cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:46.881852 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87690fa-7666-4941-88a9-4f6e8aa48cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:46.882130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87690fa-7666-4941-88a9-4f6e8aa48cca admin admin] PATH_INFO: `/v3/users/b72067d66a7649e7b8c120dd4d3fe635` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:46.896286 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87690fa-7666-4941-88a9-4f6e8aa48cca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b72067d66a7649e7b8c120dd4d3fe635)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:46.898451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87690fa-7666-4941-88a9-4f6e8aa48cca admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:47.139353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a87690fa-7666-4941-88a9-4f6e8aa48cca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b72067d66a7649e7b8c120dd4d3fe635'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:47.146153 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a87690fa-7666-4941-88a9-4f6e8aa48cca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b72067d66a7649e7b8c120dd4d3fe635'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:47.146800 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a87690fa-7666-4941-88a9-4f6e8aa48cca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b72067d66a7649e7b8c120dd4d3fe635'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:47.147439 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a87690fa-7666-4941-88a9-4f6e8aa48cca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b72067d66a7649e7b8c120dd4d3fe635'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:47.159373 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a87690fa-7666-4941-88a9-4f6e8aa48cca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b72067d66a7649e7b8c120dd4d3fe635'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:47.244435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1340/2714] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:19:46 2022] DELETE /identity/v3/users/b72067d66a7649e7b8c120dd4d3fe635 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:47.503183 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad9656-dfe4-48a0-ad8c-ab311cb6c19d service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:47.514465 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad9656-dfe4-48a0-ad8c-ab311cb6c19d service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:47.574536 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728dee3-59c8-437d-b093-6b77575c3007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:47.575976 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728dee3-59c8-437d-b093-6b77575c3007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:47.576419 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728dee3-59c8-437d-b093-6b77575c3007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:47.576604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728dee3-59c8-437d-b093-6b77575c3007 admin admin] PATH_INFO: `/v3/projects/5eb090b54f744ce5bfd44a2fa18644b7` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:47.588567 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728dee3-59c8-437d-b093-6b77575c3007 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eb090b54f744ce5bfd44a2fa18644b7)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:47.592065 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728dee3-59c8-437d-b093-6b77575c3007 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:47.727962 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3728dee3-59c8-437d-b093-6b77575c3007 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eb090b54f744ce5bfd44a2fa18644b7', 'request_id': 'req-3728dee3-59c8-437d-b093-6b77575c3007'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:47.806263 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3728dee3-59c8-437d-b093-6b77575c3007 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eb090b54f744ce5bfd44a2fa18644b7', 'request_id': 'req-3728dee3-59c8-437d-b093-6b77575c3007'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:47.812219 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3728dee3-59c8-437d-b093-6b77575c3007 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eb090b54f744ce5bfd44a2fa18644b7', 'request_id': 'req-3728dee3-59c8-437d-b093-6b77575c3007'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:47.824760 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3728dee3-59c8-437d-b093-6b77575c3007 admin admin] The token cache is being invalidate because project 5eb090b54f744ce5bfd44a2fa18644b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:19:47.825056 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3728dee3-59c8-437d-b093-6b77575c3007 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:47.831555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3728dee3-59c8-437d-b093-6b77575c3007 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:19:47.831555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1375/2715] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:19:47 2022] DELETE /identity/v3/projects/5eb090b54f744ce5bfd44a2fa18644b7 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:47.832670 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a0a54-1e5c-4eda-a799-0ba8f325cf95 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:47.832977 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a0a54-1e5c-4eda-a799-0ba8f325cf95 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:47.833258 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a0a54-1e5c-4eda-a799-0ba8f325cf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:47.839618 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:47.852052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-dc7a0a54-1e5c-4eda-a799-0ba8f325cf95 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:47.853263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-dc7a0a54-1e5c-4eda-a799-0ba8f325cf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:47.917206 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1341/2716] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:47.925867 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fc76c-e297-4d40-81e0-e4bfa95c87e0 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:47.931115 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fc76c-e297-4d40-81e0-e4bfa95c87e0 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:47.931115 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fc76c-e297-4d40-81e0-e4bfa95c87e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:47.944349 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-df7fc76c-e297-4d40-81e0-e4bfa95c87e0 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:47.945569 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-df7fc76c-e297-4d40-81e0-e4bfa95c87e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:47.953559 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1376/2717] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:47.957776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a0a54-1e5c-4eda-a799-0ba8f325cf95 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:47.968022 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a0a54-1e5c-4eda-a799-0ba8f325cf95 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:47.972692 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a90c39-0034-48b2-a42d-714606d584d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:47.973342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a90c39-0034-48b2-a42d-714606d584d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:47.973624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a90c39-0034-48b2-a42d-714606d584d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:47.973913 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a90c39-0034-48b2-a42d-714606d584d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:47.974619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a90c39-0034-48b2-a42d-714606d584d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:19:47.974924 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a90c39-0034-48b2-a42d-714606d584d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:47.976588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a90c39-0034-48b2-a42d-714606d584d6 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:47.989178 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1342/2718] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:19:47.993734 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fc76c-e297-4d40-81e0-e4bfa95c87e0 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:48.003584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fc76c-e297-4d40-81e0-e4bfa95c87e0 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:48.008599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6002b-5b1a-48c4-b946-21f7929b1682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:48.009277 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6002b-5b1a-48c4-b946-21f7929b1682 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:48.009570 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6002b-5b1a-48c4-b946-21f7929b1682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:48.009851 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6002b-5b1a-48c4-b946-21f7929b1682 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:48.010649 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e6002b-5b1a-48c4-b946-21f7929b1682 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:48.012648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e6002b-5b1a-48c4-b946-21f7929b1682 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:48.019412 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:48.127572 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1377/2719] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:19:47 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:19:48.131863 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a90c39-0034-48b2-a42d-714606d584d6 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:48.142424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a90c39-0034-48b2-a42d-714606d584d6 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:48.197341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13df9f-5ec2-4bb9-a0fd-120043f9d24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:48.198006 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13df9f-5ec2-4bb9-a0fd-120043f9d24d admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:48.198295 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13df9f-5ec2-4bb9-a0fd-120043f9d24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:48.198571 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13df9f-5ec2-4bb9-a0fd-120043f9d24d admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:48.199287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13df9f-5ec2-4bb9-a0fd-120043f9d24d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:48.201753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13df9f-5ec2-4bb9-a0fd-120043f9d24d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:48.310011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1343/2720] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:19:48 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:19:48.314383 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6002b-5b1a-48c4-b946-21f7929b1682 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:48.325885 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6002b-5b1a-48c4-b946-21f7929b1682 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:48.330834 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe68e7-06fc-4c91-94f8-0fb5020f5583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:48.331571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe68e7-06fc-4c91-94f8-0fb5020f5583 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:48.332120 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe68e7-06fc-4c91-94f8-0fb5020f5583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:48.332420 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe68e7-06fc-4c91-94f8-0fb5020f5583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:48.333178 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe68e7-06fc-4c91-94f8-0fb5020f5583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:48.334918 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe68e7-06fc-4c91-94f8-0fb5020f5583 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:48.346604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1378/2721] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:48.350924 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13df9f-5ec2-4bb9-a0fd-120043f9d24d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:48.361256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13df9f-5ec2-4bb9-a0fd-120043f9d24d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:48.366035 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e70c1-8f59-4310-a590-34814837cbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:48.366694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e70c1-8f59-4310-a590-34814837cbbd admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:48.366976 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e70c1-8f59-4310-a590-34814837cbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:48.367281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e70c1-8f59-4310-a590-34814837cbbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:48.370974 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e70c1-8f59-4310-a590-34814837cbbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:48.370974 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e70c1-8f59-4310-a590-34814837cbbd admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:48.379590 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1344/2722] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:19:48.383611 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe68e7-06fc-4c91-94f8-0fb5020f5583 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:48.394010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe68e7-06fc-4c91-94f8-0fb5020f5583 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:48.398663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915da9a-9ab3-4c06-9e89-f80c29d8b108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:48.411834 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915da9a-9ab3-4c06-9e89-f80c29d8b108 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:48.411834 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915da9a-9ab3-4c06-9e89-f80c29d8b108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:48.411834 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915da9a-9ab3-4c06-9e89-f80c29d8b108 admin admin] PATH_INFO: `/v3/projects/2f0e7f15120d41fba8d239623faee2ec/users/d49c23c64a684a39912c5cdfbeb1a1d5/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:48.415493 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915da9a-9ab3-4c06-9e89-f80c29d8b108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f0e7f15120d41fba8d239623faee2ec, user_id=d49c23c64a684a39912c5cdfbeb1a1d5, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:48.418133 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915da9a-9ab3-4c06-9e89-f80c29d8b108 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:48.519294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1379/2723] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:19:48 2022] PUT /identity/v3/projects/2f0e7f15120d41fba8d239623faee2ec/users/d49c23c64a684a39912c5cdfbeb1a1d5/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:19:48.524345 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b02d1-346c-4d10-8bd7-fcfe7e714ac7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:48.524639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b02d1-346c-4d10-8bd7-fcfe7e714ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:48.524923 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b02d1-346c-4d10-8bd7-fcfe7e714ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:48.542787 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-596b02d1-346c-4d10-8bd7-fcfe7e714ac7 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:48.544062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-596b02d1-346c-4d10-8bd7-fcfe7e714ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:48.602402 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1345/2724] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:48.612055 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915da9a-9ab3-4c06-9e89-f80c29d8b108 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:48.613723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a915da9a-9ab3-4c06-9e89-f80c29d8b108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:48.670891 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915da9a-9ab3-4c06-9e89-f80c29d8b108 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:48.674541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f943c28-d365-4cb9-8b0a-879f004c9517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:48.675202 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f943c28-d365-4cb9-8b0a-879f004c9517 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:48.675507 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f943c28-d365-4cb9-8b0a-879f004c9517 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:48.675773 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f943c28-d365-4cb9-8b0a-879f004c9517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:48.685006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f943c28-d365-4cb9-8b0a-879f004c9517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:48.686488 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f943c28-d365-4cb9-8b0a-879f004c9517 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:48.696006 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1380/2725] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:19:53.553619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28f354-e212-4ad9-8aea-9ea1756d73a8 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:53.554104 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28f354-e212-4ad9-8aea-9ea1756d73a8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:53.554303 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28f354-e212-4ad9-8aea-9ea1756d73a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:53.580021 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:53.589583 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ca28f354-e212-4ad9-8aea-9ea1756d73a8 None None] MFA Rules not processed for user `d49c23c64a684a39912c5cdfbeb1a1d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:53.591563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ca28f354-e212-4ad9-8aea-9ea1756d73a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:53.687470 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1346/2726] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:53.710308 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f943c28-d365-4cb9-8b0a-879f004c9517 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:53.711831 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2f943c28-d365-4cb9-8b0a-879f004c9517 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:53.788478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f943c28-d365-4cb9-8b0a-879f004c9517 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:53.794952 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd8a83-112d-4754-a965-638fe0ac4270 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:53.795651 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd8a83-112d-4754-a965-638fe0ac4270 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:53.795944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd8a83-112d-4754-a965-638fe0ac4270 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:53.796235 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd8a83-112d-4754-a965-638fe0ac4270 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:53.804832 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd8a83-112d-4754-a965-638fe0ac4270 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:53.806450 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd8a83-112d-4754-a965-638fe0ac4270 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:53.816584 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1381/2727] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:19:56.082797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0c25f-a65c-4316-a965-1b688e148db2 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:56.083317 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0c25f-a65c-4316-a965-1b688e148db2 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:56.083425 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0c25f-a65c-4316-a965-1b688e148db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:56.101496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3fa0c25f-a65c-4316-a965-1b688e148db2 None None] MFA Rules not processed for user `d49c23c64a684a39912c5cdfbeb1a1d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:19:56.102707 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3fa0c25f-a65c-4316-a965-1b688e148db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:56.110632 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1347/2728] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:19:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:19:56.452505 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd8a83-112d-4754-a965-638fe0ac4270 service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:56.453986 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-71bd8a83-112d-4754-a965-638fe0ac4270 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:56.466840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd8a83-112d-4754-a965-638fe0ac4270 service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:56.470474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb49068-29bf-4146-a5c3-0e0a7e5f0944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:56.471147 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb49068-29bf-4146-a5c3-0e0a7e5f0944 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:56.471471 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb49068-29bf-4146-a5c3-0e0a7e5f0944 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:56.471944 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb49068-29bf-4146-a5c3-0e0a7e5f0944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:56.473908 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-feb49068-29bf-4146-a5c3-0e0a7e5f0944 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:56.484462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb49068-29bf-4146-a5c3-0e0a7e5f0944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:56.485920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb49068-29bf-4146-a5c3-0e0a7e5f0944 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:56.494228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1382/2729] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:19:56.989807 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0c25f-a65c-4316-a965-1b688e148db2 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:56.993026 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3fa0c25f-a65c-4316-a965-1b688e148db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:57.126172 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0c25f-a65c-4316-a965-1b688e148db2 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:57.133248 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ed286-a5ff-48c0-9ceb-7b77d23186d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c577fa8670e34449a80de0c8e0a15a8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:57.135148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ed286-a5ff-48c0-9ceb-7b77d23186d3 service placement] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:57.136341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ed286-a5ff-48c0-9ceb-7b77d23186d3 service placement] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:57.136645 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ed286-a5ff-48c0-9ceb-7b77d23186d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:57.140052 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0a1ed286-a5ff-48c0-9ceb-7b77d23186d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:57.163440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ed286-a5ff-48c0-9ceb-7b77d23186d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:57.166430 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ed286-a5ff-48c0-9ceb-7b77d23186d3 service placement] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:57.180849 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1348/2730] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:19:59.239942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb49068-29bf-4146-a5c3-0e0a7e5f0944 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:19:59.241364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-feb49068-29bf-4146-a5c3-0e0a7e5f0944 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:59.273586 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:19:59.301606 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb49068-29bf-4146-a5c3-0e0a7e5f0944 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:19:59.305247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad1e55-0dc1-4e1e-90eb-fe2d4403ca79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:19:59.305895 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad1e55-0dc1-4e1e-90eb-fe2d4403ca79 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:19:59.306183 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad1e55-0dc1-4e1e-90eb-fe2d4403ca79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:19:59.306464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad1e55-0dc1-4e1e-90eb-fe2d4403ca79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:19:59.308248 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a8ad1e55-0dc1-4e1e-90eb-fe2d4403ca79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:19:59.368648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad1e55-0dc1-4e1e-90eb-fe2d4403ca79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:19:59.370210 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad1e55-0dc1-4e1e-90eb-fe2d4403ca79 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:19:59.379726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1383/2731] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:19:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:20:07.174868 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=63 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:20:09.416035 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=36 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:20:10.354477 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ed286-a5ff-48c0-9ceb-7b77d23186d3 service placement] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:10.356061 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0a1ed286-a5ff-48c0-9ceb-7b77d23186d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:20:10.372050 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ed286-a5ff-48c0-9ceb-7b77d23186d3 service placement] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:10.378094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c84a25-32f6-460d-819d-502eac3a8eeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:10.378745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c84a25-32f6-460d-819d-502eac3a8eeb service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:10.379034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c84a25-32f6-460d-819d-502eac3a8eeb service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:10.379354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c84a25-32f6-460d-819d-502eac3a8eeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:10.381148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c9c84a25-32f6-460d-819d-502eac3a8eeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:20:10.394664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c84a25-32f6-460d-819d-502eac3a8eeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:10.396429 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c84a25-32f6-460d-819d-502eac3a8eeb service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:10.412413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1349/2732] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:20:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:20:11.099184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f348c89-547c-4b65-b2a2-2b13364303af None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:11.100921 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f348c89-547c-4b65-b2a2-2b13364303af None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:11.100921 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f348c89-547c-4b65-b2a2-2b13364303af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:11.117731 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9f348c89-547c-4b65-b2a2-2b13364303af None None] MFA Rules not processed for user `89bf838da11b41bc9b9f77c9f10a6bbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:20:11.118960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9f348c89-547c-4b65-b2a2-2b13364303af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:20:11.126749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1384/2733] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:20:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:20:15.280115 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c84a25-32f6-460d-819d-502eac3a8eeb service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:15.293552 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c84a25-32f6-460d-819d-502eac3a8eeb service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:15.304509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5ab35-5fbb-4e18-a32b-b122f5dc803f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:15.305436 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5ab35-5fbb-4e18-a32b-b122f5dc803f service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:15.306029 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5ab35-5fbb-4e18-a32b-b122f5dc803f service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:15.306512 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5ab35-5fbb-4e18-a32b-b122f5dc803f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:15.308394 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c8f5ab35-5fbb-4e18-a32b-b122f5dc803f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:20:15.323590 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5ab35-5fbb-4e18-a32b-b122f5dc803f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:15.326292 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5ab35-5fbb-4e18-a32b-b122f5dc803f service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:15.338128 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1350/2734] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:20:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:20:21.112285 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:20:25.333000 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=12 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:20:31.479142 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f348c89-547c-4b65-b2a2-2b13364303af None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:31.489290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f348c89-547c-4b65-b2a2-2b13364303af None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:31.493141 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097b689-7186-461c-b8fb-22fc7a0b4983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:31.493775 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097b689-7186-461c-b8fb-22fc7a0b4983 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:31.494059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097b689-7186-461c-b8fb-22fc7a0b4983 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:31.494335 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097b689-7186-461c-b8fb-22fc7a0b4983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:31.496130 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6097b689-7186-461c-b8fb-22fc7a0b4983 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:20:31.561743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097b689-7186-461c-b8fb-22fc7a0b4983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:31.563340 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097b689-7186-461c-b8fb-22fc7a0b4983 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:31.573894 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1385/2735] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:20:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:20:35.477029 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5ab35-5fbb-4e18-a32b-b122f5dc803f service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:35.478446 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c8f5ab35-5fbb-4e18-a32b-b122f5dc803f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:20:35.491414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5ab35-5fbb-4e18-a32b-b122f5dc803f service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:35.495885 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb6a73-aa3e-4b4c-ab34-5e44f97ea901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:35.496735 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb6a73-aa3e-4b4c-ab34-5e44f97ea901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:35.497529 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb6a73-aa3e-4b4c-ab34-5e44f97ea901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:35.497802 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb6a73-aa3e-4b4c-ab34-5e44f97ea901 admin admin] PATH_INFO: `/v3/users/c65f75c2f1f54ec28dcc4bb3fd3b818a` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:35.498975 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb6a73-aa3e-4b4c-ab34-5e44f97ea901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c65f75c2f1f54ec28dcc4bb3fd3b818a)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:35.500918 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb6a73-aa3e-4b4c-ab34-5e44f97ea901 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:35.691047 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-35eb6a73-aa3e-4b4c-ab34-5e44f97ea901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c65f75c2f1f54ec28dcc4bb3fd3b818a'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:20:35.697831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-35eb6a73-aa3e-4b4c-ab34-5e44f97ea901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c65f75c2f1f54ec28dcc4bb3fd3b818a'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:20:35.698490 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-35eb6a73-aa3e-4b4c-ab34-5e44f97ea901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c65f75c2f1f54ec28dcc4bb3fd3b818a'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:20:35.699100 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-35eb6a73-aa3e-4b4c-ab34-5e44f97ea901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c65f75c2f1f54ec28dcc4bb3fd3b818a'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:20:35.711071 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-35eb6a73-aa3e-4b4c-ab34-5e44f97ea901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c65f75c2f1f54ec28dcc4bb3fd3b818a'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:20:35.780967 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1351/2736] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:20:35 2022] DELETE /identity/v3/users/c65f75c2f1f54ec28dcc4bb3fd3b818a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:20:35.785764 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1f886-1cce-44ae-a525-a33935dd11b3 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:35.786318 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1f886-1cce-44ae-a525-a33935dd11b3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:35.786424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1f886-1cce-44ae-a525-a33935dd11b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:35.803579 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-80f1f886-1cce-44ae-a525-a33935dd11b3 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:20:35.804809 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-80f1f886-1cce-44ae-a525-a33935dd11b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:20:35.864374 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1386/2737] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:20:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:20:35.874823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb6a73-aa3e-4b4c-ab34-5e44f97ea901 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:35.876296 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-35eb6a73-aa3e-4b4c-ab34-5e44f97ea901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:20:35.932735 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb6a73-aa3e-4b4c-ab34-5e44f97ea901 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:35.936379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496ee5e-c077-4b46-8063-8f189164f256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:35.937021 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496ee5e-c077-4b46-8063-8f189164f256 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:35.937312 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496ee5e-c077-4b46-8063-8f189164f256 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:35.937586 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496ee5e-c077-4b46-8063-8f189164f256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:35.946657 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496ee5e-c077-4b46-8063-8f189164f256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:35.948061 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496ee5e-c077-4b46-8063-8f189164f256 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:35.957980 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1352/2738] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:20:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:20:36.367825 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1f886-1cce-44ae-a525-a33935dd11b3 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:36.369307 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-80f1f886-1cce-44ae-a525-a33935dd11b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:20:36.382647 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1f886-1cce-44ae-a525-a33935dd11b3 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:36.388066 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069be0d5-0c6b-4130-ad6b-db23a8541dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:36.388948 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069be0d5-0c6b-4130-ad6b-db23a8541dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:36.389263 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069be0d5-0c6b-4130-ad6b-db23a8541dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:36.389541 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069be0d5-0c6b-4130-ad6b-db23a8541dc3 admin admin] PATH_INFO: `/v3/projects/8e5c89bd4ec644daa5ffff4767c540ec` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:36.390991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069be0d5-0c6b-4130-ad6b-db23a8541dc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e5c89bd4ec644daa5ffff4767c540ec)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:36.394045 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069be0d5-0c6b-4130-ad6b-db23a8541dc3 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:36.508634 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-069be0d5-0c6b-4130-ad6b-db23a8541dc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e5c89bd4ec644daa5ffff4767c540ec', 'request_id': 'req-069be0d5-0c6b-4130-ad6b-db23a8541dc3'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:20:36.571010 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-069be0d5-0c6b-4130-ad6b-db23a8541dc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e5c89bd4ec644daa5ffff4767c540ec', 'request_id': 'req-069be0d5-0c6b-4130-ad6b-db23a8541dc3'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:20:36.576741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-069be0d5-0c6b-4130-ad6b-db23a8541dc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e5c89bd4ec644daa5ffff4767c540ec', 'request_id': 'req-069be0d5-0c6b-4130-ad6b-db23a8541dc3'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:20:36.587580 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-069be0d5-0c6b-4130-ad6b-db23a8541dc3 admin admin] The token cache is being invalidate because project 8e5c89bd4ec644daa5ffff4767c540ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:20:36.587867 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-069be0d5-0c6b-4130-ad6b-db23a8541dc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:20:36.588656 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-069be0d5-0c6b-4130-ad6b-db23a8541dc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:20:36.590036 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1387/2739] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:20:36 2022] DELETE /identity/v3/projects/8e5c89bd4ec644daa5ffff4767c540ec => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:20:36.595871 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe27a98-3a3b-4969-80ce-3662da023d04 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:36.596376 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe27a98-3a3b-4969-80ce-3662da023d04 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:36.596479 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe27a98-3a3b-4969-80ce-3662da023d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:36.615558 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-dfe27a98-3a3b-4969-80ce-3662da023d04 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:20:36.616766 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-dfe27a98-3a3b-4969-80ce-3662da023d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:20:36.675358 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1353/2740] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:20:36.683073 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440caf5-5621-4413-ae83-0b5a353b793b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:36.683396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440caf5-5621-4413-ae83-0b5a353b793b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:36.683680 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440caf5-5621-4413-ae83-0b5a353b793b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:36.701885 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c440caf5-5621-4413-ae83-0b5a353b793b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:20:36.703070 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c440caf5-5621-4413-ae83-0b5a353b793b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:20:36.710805 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1388/2741] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:20:36.714748 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe27a98-3a3b-4969-80ce-3662da023d04 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:36.724338 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe27a98-3a3b-4969-80ce-3662da023d04 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:36.728934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edcb64-790f-43c3-87d9-4d6b27801f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:36.729619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edcb64-790f-43c3-87d9-4d6b27801f8c admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:36.729909 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edcb64-790f-43c3-87d9-4d6b27801f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:36.730190 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edcb64-790f-43c3-87d9-4d6b27801f8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:36.730917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11edcb64-790f-43c3-87d9-4d6b27801f8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:20:36.731783 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11edcb64-790f-43c3-87d9-4d6b27801f8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:36.733393 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11edcb64-790f-43c3-87d9-4d6b27801f8c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:36.744932 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1354/2742] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 19:20:36.749683 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440caf5-5621-4413-ae83-0b5a353b793b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:36.759152 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440caf5-5621-4413-ae83-0b5a353b793b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:36.766097 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe35cce-585b-418e-bd53-b469ce320369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:36.766763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe35cce-585b-418e-bd53-b469ce320369 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:36.767045 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe35cce-585b-418e-bd53-b469ce320369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:36.767350 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe35cce-585b-418e-bd53-b469ce320369 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:36.768043 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe35cce-585b-418e-bd53-b469ce320369 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:36.769983 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe35cce-585b-418e-bd53-b469ce320369 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:36.779432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:20:36.856787 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1389/2743] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:20:36 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:20:36.860754 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edcb64-790f-43c3-87d9-4d6b27801f8c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:36.865829 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:20:36.871092 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edcb64-790f-43c3-87d9-4d6b27801f8c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:36.928371 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722150b-1ae1-44a1-96ae-d46b9954e270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:36.929045 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722150b-1ae1-44a1-96ae-d46b9954e270 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:36.929335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722150b-1ae1-44a1-96ae-d46b9954e270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:36.929611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722150b-1ae1-44a1-96ae-d46b9954e270 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:36.930305 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722150b-1ae1-44a1-96ae-d46b9954e270 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:36.932288 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722150b-1ae1-44a1-96ae-d46b9954e270 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:37.000517 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1355/2744] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:20:36 2022] POST /identity/v3/users => generated 286 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:20:37.004532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe35cce-585b-418e-bd53-b469ce320369 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:37.015982 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe35cce-585b-418e-bd53-b469ce320369 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:37.020505 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668ed5d-6259-40ab-a769-dd25f917aed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:37.021161 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668ed5d-6259-40ab-a769-dd25f917aed8 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:37.021444 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668ed5d-6259-40ab-a769-dd25f917aed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:37.021720 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668ed5d-6259-40ab-a769-dd25f917aed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:37.022438 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668ed5d-6259-40ab-a769-dd25f917aed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:37.024067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668ed5d-6259-40ab-a769-dd25f917aed8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:37.034333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1390/2745] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:20:37.038215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722150b-1ae1-44a1-96ae-d46b9954e270 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:37.048152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722150b-1ae1-44a1-96ae-d46b9954e270 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:37.052758 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ae10c-2ad7-489f-ad17-66e131b897bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:37.053412 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ae10c-2ad7-489f-ad17-66e131b897bf admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:37.053701 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ae10c-2ad7-489f-ad17-66e131b897bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:37.053979 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ae10c-2ad7-489f-ad17-66e131b897bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:37.054697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ae10c-2ad7-489f-ad17-66e131b897bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:37.056311 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ae10c-2ad7-489f-ad17-66e131b897bf admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:37.067145 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1356/2746] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:20:37.071088 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668ed5d-6259-40ab-a769-dd25f917aed8 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:37.081469 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668ed5d-6259-40ab-a769-dd25f917aed8 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:37.086382 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac1fc1-1b09-4f21-8e6e-0dd5474867fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:37.087316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac1fc1-1b09-4f21-8e6e-0dd5474867fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:37.087604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac1fc1-1b09-4f21-8e6e-0dd5474867fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:37.087888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac1fc1-1b09-4f21-8e6e-0dd5474867fd admin admin] PATH_INFO: `/v3/projects/97d0d5140e26495f856abe6dc4ce8549/users/a38c1207b03d4125bcbe99c569573dc6/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:37.104700 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac1fc1-1b09-4f21-8e6e-0dd5474867fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=97d0d5140e26495f856abe6dc4ce8549, user_id=a38c1207b03d4125bcbe99c569573dc6, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:37.108087 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac1fc1-1b09-4f21-8e6e-0dd5474867fd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:37.188484 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1391/2747] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:20:37 2022] PUT /identity/v3/projects/97d0d5140e26495f856abe6dc4ce8549/users/a38c1207b03d4125bcbe99c569573dc6/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:20:37.193611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bb7c0-052b-4b6b-a463-22e458ec8ed2 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:37.195415 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bb7c0-052b-4b6b-a463-22e458ec8ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:37.195415 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bb7c0-052b-4b6b-a463-22e458ec8ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:37.211887 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7b7bb7c0-052b-4b6b-a463-22e458ec8ed2 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:20:37.213090 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7b7bb7c0-052b-4b6b-a463-22e458ec8ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:20:37.270464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1357/2748] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:20:37.280665 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac1fc1-1b09-4f21-8e6e-0dd5474867fd admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:37.282320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d8ac1fc1-1b09-4f21-8e6e-0dd5474867fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:20:37.337326 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac1fc1-1b09-4f21-8e6e-0dd5474867fd admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:37.341529 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300be06-4763-4f08-b4aa-53c889b903d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:37.342176 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300be06-4763-4f08-b4aa-53c889b903d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:37.342464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300be06-4763-4f08-b4aa-53c889b903d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:37.342741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300be06-4763-4f08-b4aa-53c889b903d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:37.351742 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300be06-4763-4f08-b4aa-53c889b903d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:37.353184 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300be06-4763-4f08-b4aa-53c889b903d2 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:37.363075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1392/2749] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:20:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:20:42.567901 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd90511-1c81-4015-b34e-2201fbf47189 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:42.569456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd90511-1c81-4015-b34e-2201fbf47189 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:42.569456 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd90511-1c81-4015-b34e-2201fbf47189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:42.600361 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ccd90511-1c81-4015-b34e-2201fbf47189 None None] MFA Rules not processed for user `a38c1207b03d4125bcbe99c569573dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:20:42.601859 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ccd90511-1c81-4015-b34e-2201fbf47189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:20:42.684930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1358/2750] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:20:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:20:42.685275 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300be06-4763-4f08-b4aa-53c889b903d2 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:42.694093 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300be06-4763-4f08-b4aa-53c889b903d2 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:42.699516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2dcd2-7b50-4a21-b7be-91020c23af5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:42.700442 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2dcd2-7b50-4a21-b7be-91020c23af5d admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:42.700731 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2dcd2-7b50-4a21-b7be-91020c23af5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:42.701181 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2dcd2-7b50-4a21-b7be-91020c23af5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:42.701899 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2dcd2-7b50-4a21-b7be-91020c23af5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:42.703871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2dcd2-7b50-4a21-b7be-91020c23af5d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:42.812109 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1393/2751] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:20:42 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:20:42.816101 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd90511-1c81-4015-b34e-2201fbf47189 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:42.828005 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd90511-1c81-4015-b34e-2201fbf47189 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:42.838202 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:20:42.904546 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016c117-caf2-4396-87f3-4ced90807c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:42.905224 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016c117-caf2-4396-87f3-4ced90807c13 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:42.905515 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016c117-caf2-4396-87f3-4ced90807c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:42.905818 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016c117-caf2-4396-87f3-4ced90807c13 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:42.906546 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016c117-caf2-4396-87f3-4ced90807c13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:42.908613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016c117-caf2-4396-87f3-4ced90807c13 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:43.012071 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1359/2752] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:20:42 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:20:43.016688 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2dcd2-7b50-4a21-b7be-91020c23af5d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:43.027998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2dcd2-7b50-4a21-b7be-91020c23af5d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:43.032771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e0883-40ee-4c16-8f63-18d238556a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:43.033477 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e0883-40ee-4c16-8f63-18d238556a9e admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:43.033804 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e0883-40ee-4c16-8f63-18d238556a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:43.034088 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e0883-40ee-4c16-8f63-18d238556a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:43.034814 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e0883-40ee-4c16-8f63-18d238556a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:43.036531 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e0883-40ee-4c16-8f63-18d238556a9e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:43.048348 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1394/2753] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:20:43.052531 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016c117-caf2-4396-87f3-4ced90807c13 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:43.063379 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016c117-caf2-4396-87f3-4ced90807c13 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:43.068168 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60549b-3df7-494b-a969-ac48a60f9a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:43.069092 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60549b-3df7-494b-a969-ac48a60f9a9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:43.069378 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60549b-3df7-494b-a969-ac48a60f9a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:43.069655 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60549b-3df7-494b-a969-ac48a60f9a9d admin admin] PATH_INFO: `/v3/projects/d9180ad54c1e4532a67d99621f6363b2/users/6d83b42d8333430d9cc00a3afedf9304/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:43.086837 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60549b-3df7-494b-a969-ac48a60f9a9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9180ad54c1e4532a67d99621f6363b2, user_id=6d83b42d8333430d9cc00a3afedf9304, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:43.089717 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60549b-3df7-494b-a969-ac48a60f9a9d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:43.183768 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1360/2754] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:20:43 2022] PUT /identity/v3/projects/d9180ad54c1e4532a67d99621f6363b2/users/6d83b42d8333430d9cc00a3afedf9304/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:20:43.188174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e0883-40ee-4c16-8f63-18d238556a9e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:43.198629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e0883-40ee-4c16-8f63-18d238556a9e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:43.257778 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28510141-9c14-4002-bc82-e0dadb2a2e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:43.258464 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28510141-9c14-4002-bc82-e0dadb2a2e0c admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:43.258749 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28510141-9c14-4002-bc82-e0dadb2a2e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:43.259024 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28510141-9c14-4002-bc82-e0dadb2a2e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:43.259835 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28510141-9c14-4002-bc82-e0dadb2a2e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:43.261474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28510141-9c14-4002-bc82-e0dadb2a2e0c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:43.272663 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1395/2755] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:20:43.276813 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60549b-3df7-494b-a969-ac48a60f9a9d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:43.287848 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60549b-3df7-494b-a969-ac48a60f9a9d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:43.292858 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dd4cb-9506-437d-bc8d-a16e1533e9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:43.293861 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dd4cb-9506-437d-bc8d-a16e1533e9db admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:43.294235 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dd4cb-9506-437d-bc8d-a16e1533e9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:43.294582 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dd4cb-9506-437d-bc8d-a16e1533e9db admin admin] PATH_INFO: `/v3/projects/d9180ad54c1e4532a67d99621f6363b2/users/6d83b42d8333430d9cc00a3afedf9304/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:43.296722 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279dd4cb-9506-437d-bc8d-a16e1533e9db admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9180ad54c1e4532a67d99621f6363b2, user_id=6d83b42d8333430d9cc00a3afedf9304, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:43.299532 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279dd4cb-9506-437d-bc8d-a16e1533e9db admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:43.311236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1361/2756] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:20:43 2022] PUT /identity/v3/projects/d9180ad54c1e4532a67d99621f6363b2/users/6d83b42d8333430d9cc00a3afedf9304/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:20:48.764051 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547544e-9139-437a-87fc-35a90b4480c8 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:48.764564 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547544e-9139-437a-87fc-35a90b4480c8 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:48.764564 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547544e-9139-437a-87fc-35a90b4480c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:48.791867 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7547544e-9139-437a-87fc-35a90b4480c8 None None] MFA Rules not processed for user `6d83b42d8333430d9cc00a3afedf9304`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:20:48.793015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7547544e-9139-437a-87fc-35a90b4480c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:20:48.801473 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:20:48.882279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1396/2757] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:20:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:20:48.899084 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dd4cb-9506-437d-bc8d-a16e1533e9db admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:20:48.900917 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-279dd4cb-9506-437d-bc8d-a16e1533e9db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:20:48.979824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dd4cb-9506-437d-bc8d-a16e1533e9db admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:20:48.986018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e621791-bbcf-42de-bd84-687dcb51bf51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:20:48.986677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e621791-bbcf-42de-bd84-687dcb51bf51 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:48.986969 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e621791-bbcf-42de-bd84-687dcb51bf51 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:48.987281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e621791-bbcf-42de-bd84-687dcb51bf51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:49.070326 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e621791-bbcf-42de-bd84-687dcb51bf51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:20:49.072223 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e621791-bbcf-42de-bd84-687dcb51bf51 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:20:49.077943 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:20:49.083521 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1362/2758] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:20:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:20:51.635476 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc28f8-06b3-42fd-9601-1c7544c72237 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:20:51.635960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc28f8-06b3-42fd-9601-1c7544c72237 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:20:51.636093 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc28f8-06b3-42fd-9601-1c7544c72237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:20:51.656686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5fdc28f8-06b3-42fd-9601-1c7544c72237 None None] MFA Rules not processed for user `a38c1207b03d4125bcbe99c569573dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:20:51.657909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5fdc28f8-06b3-42fd-9601-1c7544c72237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:20:51.669960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1397/2759] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:20:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:21:01.650676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=28 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:21:14.840162 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e621791-bbcf-42de-bd84-687dcb51bf51 service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:21:14.893514 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e621791-bbcf-42de-bd84-687dcb51bf51 service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:21:14.897200 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e2de2-f4b8-406a-9a58-440e6ba3d146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:21:14.897839 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e2de2-f4b8-406a-9a58-440e6ba3d146 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:21:14.898124 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e2de2-f4b8-406a-9a58-440e6ba3d146 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:21:14.898404 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e2de2-f4b8-406a-9a58-440e6ba3d146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:21:14.900174 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d59e2de2-f4b8-406a-9a58-440e6ba3d146 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:21:14.910613 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e2de2-f4b8-406a-9a58-440e6ba3d146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:21:14.912015 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e2de2-f4b8-406a-9a58-440e6ba3d146 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:21:14.919958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1363/2760] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:21:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:21:16.931924 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc28f8-06b3-42fd-9601-1c7544c72237 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:21:16.933366 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5fdc28f8-06b3-42fd-9601-1c7544c72237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:21:17.001328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc28f8-06b3-42fd-9601-1c7544c72237 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:21:17.006350 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684d4ca-28a5-44d1-bdbe-f12a09a054cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c577fa8670e34449a80de0c8e0a15a8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:21:17.006937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684d4ca-28a5-44d1-bdbe-f12a09a054cc service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:21:17.007255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684d4ca-28a5-44d1-bdbe-f12a09a054cc service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:21:17.007537 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684d4ca-28a5-44d1-bdbe-f12a09a054cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:21:17.009328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8684d4ca-28a5-44d1-bdbe-f12a09a054cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:21:17.021493 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684d4ca-28a5-44d1-bdbe-f12a09a054cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:21:17.023064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684d4ca-28a5-44d1-bdbe-f12a09a054cc service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:21:17.033208 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1398/2761] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:21:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:21:24.916299 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=26 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:21:27.028816 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=32 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:21:34.414422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e2de2-f4b8-406a-9a58-440e6ba3d146 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:21:34.423505 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e2de2-f4b8-406a-9a58-440e6ba3d146 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:21:34.427071 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d047c3e-9751-4eb7-8f13-fc1c0c091dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:21:34.427741 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d047c3e-9751-4eb7-8f13-fc1c0c091dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:21:34.428030 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d047c3e-9751-4eb7-8f13-fc1c0c091dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:21:34.428309 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d047c3e-9751-4eb7-8f13-fc1c0c091dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:21:34.437034 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d047c3e-9751-4eb7-8f13-fc1c0c091dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:21:34.439089 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d047c3e-9751-4eb7-8f13-fc1c0c091dc9 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:21:34.448402 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1364/2762] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:21:34 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:21:44.444256 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=6 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:21:57.479290 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c079a1c-714e-4ef9-9374-979a7c2c840b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:21:57.479846 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c079a1c-714e-4ef9-9374-979a7c2c840b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:21:57.479846 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c079a1c-714e-4ef9-9374-979a7c2c840b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:21:57.497555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2c079a1c-714e-4ef9-9374-979a7c2c840b None None] MFA Rules not processed for user `d49c23c64a684a39912c5cdfbeb1a1d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:21:57.498806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2c079a1c-714e-4ef9-9374-979a7c2c840b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:21:57.552762 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1399/2763] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:21:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:22:07.546118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=25 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:22:50.234228 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d047c3e-9751-4eb7-8f13-fc1c0c091dc9 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:22:50.243626 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d047c3e-9751-4eb7-8f13-fc1c0c091dc9 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:22:50.247269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101aab7-e1d1-415f-8d8e-8a959fe533c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:22:50.247908 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101aab7-e1d1-415f-8d8e-8a959fe533c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:22:50.248212 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101aab7-e1d1-415f-8d8e-8a959fe533c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:22:50.248471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101aab7-e1d1-415f-8d8e-8a959fe533c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:22:50.250247 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0101aab7-e1d1-415f-8d8e-8a959fe533c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:22:50.314993 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101aab7-e1d1-415f-8d8e-8a959fe533c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:22:50.316451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101aab7-e1d1-415f-8d8e-8a959fe533c9 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:22:50.325964 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1365/2764] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:22:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:22:53.834706 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c079a1c-714e-4ef9-9374-979a7c2c840b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:22:53.847246 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c079a1c-714e-4ef9-9374-979a7c2c840b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:22:53.853035 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c1cf0-9210-48f5-a9c0-66d49bb1300a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c577fa8670e34449a80de0c8e0a15a8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:22:53.853691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c1cf0-9210-48f5-a9c0-66d49bb1300a service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:22:53.854030 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c1cf0-9210-48f5-a9c0-66d49bb1300a service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:22:53.854313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c1cf0-9210-48f5-a9c0-66d49bb1300a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:22:53.856167 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-240c1cf0-9210-48f5-a9c0-66d49bb1300a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:22:53.868375 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c1cf0-9210-48f5-a9c0-66d49bb1300a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:22:53.869982 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c1cf0-9210-48f5-a9c0-66d49bb1300a service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:22:53.880368 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1400/2765] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:22:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:22:57.587710 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101aab7-e1d1-415f-8d8e-8a959fe533c9 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:22:57.589114 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0101aab7-e1d1-415f-8d8e-8a959fe533c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:22:57.602485 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101aab7-e1d1-415f-8d8e-8a959fe533c9 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:22:57.607013 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7d710-59cc-4e86-a0f7-30294c4eccf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:22:57.607987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7d710-59cc-4e86-a0f7-30294c4eccf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:22:57.608274 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7d710-59cc-4e86-a0f7-30294c4eccf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:22:57.608551 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7d710-59cc-4e86-a0f7-30294c4eccf3 admin admin] PATH_INFO: `/v3/users/89bf838da11b41bc9b9f77c9f10a6bbd` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:22:57.609733 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae7d710-59cc-4e86-a0f7-30294c4eccf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89bf838da11b41bc9b9f77c9f10a6bbd)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:22:57.611857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae7d710-59cc-4e86-a0f7-30294c4eccf3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:22:57.828822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6ae7d710-59cc-4e86-a0f7-30294c4eccf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89bf838da11b41bc9b9f77c9f10a6bbd'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:57.835654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6ae7d710-59cc-4e86-a0f7-30294c4eccf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89bf838da11b41bc9b9f77c9f10a6bbd'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:57.836318 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6ae7d710-59cc-4e86-a0f7-30294c4eccf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89bf838da11b41bc9b9f77c9f10a6bbd'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:57.836925 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6ae7d710-59cc-4e86-a0f7-30294c4eccf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89bf838da11b41bc9b9f77c9f10a6bbd'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:57.851284 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6ae7d710-59cc-4e86-a0f7-30294c4eccf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89bf838da11b41bc9b9f77c9f10a6bbd'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:57.929752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1366/2766] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:22:57 2022] DELETE /identity/v3/users/89bf838da11b41bc9b9f77c9f10a6bbd => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:22:58.154333 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c1cf0-9210-48f5-a9c0-66d49bb1300a service placement] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:22:58.166272 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c1cf0-9210-48f5-a9c0-66d49bb1300a service placement] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:22:58.224846 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a5337-fd5a-47f9-98b5-4622e7fac23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:22:58.225714 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a5337-fd5a-47f9-98b5-4622e7fac23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:22:58.226000 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a5337-fd5a-47f9-98b5-4622e7fac23b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:22:58.226274 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a5337-fd5a-47f9-98b5-4622e7fac23b admin admin] PATH_INFO: `/v3/projects/329192399b054c01a7f28f4c6fb4d484` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:22:58.227475 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a5337-fd5a-47f9-98b5-4622e7fac23b admin admin] RBAC: Authorizing `identity:delete_project(project_id=329192399b054c01a7f28f4c6fb4d484)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:22:58.229376 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a5337-fd5a-47f9-98b5-4622e7fac23b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:22:58.320215 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-938a5337-fd5a-47f9-98b5-4622e7fac23b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '329192399b054c01a7f28f4c6fb4d484', 'request_id': 'req-938a5337-fd5a-47f9-98b5-4622e7fac23b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:58.501139 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-938a5337-fd5a-47f9-98b5-4622e7fac23b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '329192399b054c01a7f28f4c6fb4d484', 'request_id': 'req-938a5337-fd5a-47f9-98b5-4622e7fac23b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:58.507412 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-938a5337-fd5a-47f9-98b5-4622e7fac23b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '329192399b054c01a7f28f4c6fb4d484', 'request_id': 'req-938a5337-fd5a-47f9-98b5-4622e7fac23b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:58.518955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-938a5337-fd5a-47f9-98b5-4622e7fac23b admin admin] The token cache is being invalidate because project 329192399b054c01a7f28f4c6fb4d484 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:22:58.519272 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-938a5337-fd5a-47f9-98b5-4622e7fac23b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:58.520089 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-938a5337-fd5a-47f9-98b5-4622e7fac23b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:58.522015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1401/2767] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:22:58 2022] DELETE /identity/v3/projects/329192399b054c01a7f28f4c6fb4d484 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:22:58.525910 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7d710-59cc-4e86-a0f7-30294c4eccf3 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:22:58.527361 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6ae7d710-59cc-4e86-a0f7-30294c4eccf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:22:58.541630 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7d710-59cc-4e86-a0f7-30294c4eccf3 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:22:58.599267 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c7629-eb81-4c4f-8279-bce18524defb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:22:58.600148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c7629-eb81-4c4f-8279-bce18524defb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:22:58.600432 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c7629-eb81-4c4f-8279-bce18524defb admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:22:58.600708 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c7629-eb81-4c4f-8279-bce18524defb admin admin] PATH_INFO: `/v3/users/a869e695cc9045988ecbf179be72d459` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:22:58.601898 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c7629-eb81-4c4f-8279-bce18524defb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a869e695cc9045988ecbf179be72d459)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:22:58.603874 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c7629-eb81-4c4f-8279-bce18524defb admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:22:58.609113 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:22:58.777706 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6e9c7629-eb81-4c4f-8279-bce18524defb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a869e695cc9045988ecbf179be72d459'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:58.784749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6e9c7629-eb81-4c4f-8279-bce18524defb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a869e695cc9045988ecbf179be72d459'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:58.785459 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6e9c7629-eb81-4c4f-8279-bce18524defb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a869e695cc9045988ecbf179be72d459'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:58.786564 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6e9c7629-eb81-4c4f-8279-bce18524defb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a869e695cc9045988ecbf179be72d459'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:58.798202 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6e9c7629-eb81-4c4f-8279-bce18524defb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a869e695cc9045988ecbf179be72d459'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:58.849454 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1367/2768] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:22:58 2022] DELETE /identity/v3/users/a869e695cc9045988ecbf179be72d459 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:22:59.074315 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a5337-fd5a-47f9-98b5-4622e7fac23b admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:22:59.082710 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-938a5337-fd5a-47f9-98b5-4622e7fac23b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:22:59.089381 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a5337-fd5a-47f9-98b5-4622e7fac23b admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:22:59.145159 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512f43d-a943-4936-8a38-449023f82da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:22:59.146525 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512f43d-a943-4936-8a38-449023f82da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:22:59.146806 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512f43d-a943-4936-8a38-449023f82da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:22:59.147080 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512f43d-a943-4936-8a38-449023f82da8 admin admin] PATH_INFO: `/v3/projects/741d8a8b02c949dab983ed7f56f6c2ac` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:22:59.148319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512f43d-a943-4936-8a38-449023f82da8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=741d8a8b02c949dab983ed7f56f6c2ac)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:22:59.150220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512f43d-a943-4936-8a38-449023f82da8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:22:59.170870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:22:59.283115 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a512f43d-a943-4936-8a38-449023f82da8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '741d8a8b02c949dab983ed7f56f6c2ac', 'request_id': 'req-a512f43d-a943-4936-8a38-449023f82da8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:59.323478 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a512f43d-a943-4936-8a38-449023f82da8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '741d8a8b02c949dab983ed7f56f6c2ac', 'request_id': 'req-a512f43d-a943-4936-8a38-449023f82da8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:59.329032 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a512f43d-a943-4936-8a38-449023f82da8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '741d8a8b02c949dab983ed7f56f6c2ac', 'request_id': 'req-a512f43d-a943-4936-8a38-449023f82da8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:59.341044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a512f43d-a943-4936-8a38-449023f82da8 admin admin] The token cache is being invalidate because project 741d8a8b02c949dab983ed7f56f6c2ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:22:59.341363 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a512f43d-a943-4936-8a38-449023f82da8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:59.342008 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a512f43d-a943-4936-8a38-449023f82da8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:22:59.343574 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1402/2769] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:22:59 2022] DELETE /identity/v3/projects/741d8a8b02c949dab983ed7f56f6c2ac => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:23:08.845304 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:23:09.339741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:23:27.411683 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c7629-eb81-4c4f-8279-bce18524defb admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:23:27.413499 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6e9c7629-eb81-4c4f-8279-bce18524defb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:23:27.469937 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c7629-eb81-4c4f-8279-bce18524defb admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:23:27.517104 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc94f1-5a2f-4c28-b049-981c2d25e064 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:23:27.517749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc94f1-5a2f-4c28-b049-981c2d25e064 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:23:27.518033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc94f1-5a2f-4c28-b049-981c2d25e064 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:23:27.518313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc94f1-5a2f-4c28-b049-981c2d25e064 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:23:27.520183 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-47cc94f1-5a2f-4c28-b049-981c2d25e064 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:23:27.590662 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc94f1-5a2f-4c28-b049-981c2d25e064 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:23:27.592136 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc94f1-5a2f-4c28-b049-981c2d25e064 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:23:27.603000 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1368/2770] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:23:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:23:29.206825 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512f43d-a943-4936-8a38-449023f82da8 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:23:29.208287 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a512f43d-a943-4936-8a38-449023f82da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:23:29.279063 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512f43d-a943-4936-8a38-449023f82da8 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:23:29.282649 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c4317-fbb2-4970-ab78-e68fd578ac9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:23:29.283331 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c4317-fbb2-4970-ab78-e68fd578ac9a service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:23:29.283615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c4317-fbb2-4970-ab78-e68fd578ac9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:23:29.283901 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c4317-fbb2-4970-ab78-e68fd578ac9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:23:29.285729 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-735c4317-fbb2-4970-ab78-e68fd578ac9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:23:29.299741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c4317-fbb2-4970-ab78-e68fd578ac9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:23:29.301170 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c4317-fbb2-4970-ab78-e68fd578ac9a service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:23:29.312703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1403/2771] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:23:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:23:37.598175 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=74 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:23:39.307905 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=26 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:23:44.187783 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef0b79-26b6-4cf7-baeb-cded20a08ea5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:23:44.188282 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef0b79-26b6-4cf7-baeb-cded20a08ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:23:44.188420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef0b79-26b6-4cf7-baeb-cded20a08ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:23:44.205997 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f2ef0b79-26b6-4cf7-baeb-cded20a08ea5 None None] MFA Rules not processed for user `d49c23c64a684a39912c5cdfbeb1a1d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:23:44.207200 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f2ef0b79-26b6-4cf7-baeb-cded20a08ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:23:44.261409 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1369/2772] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:23:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:23:54.253890 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=25 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:24:04.126167 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c4317-fbb2-4970-ab78-e68fd578ac9a service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:24:04.134621 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c4317-fbb2-4970-ab78-e68fd578ac9a service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:24:04.138142 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1f070-e3a5-4a05-8fe0-babb4318adc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:24:04.138772 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1f070-e3a5-4a05-8fe0-babb4318adc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:24:04.139061 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1f070-e3a5-4a05-8fe0-babb4318adc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:24:04.139366 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1f070-e3a5-4a05-8fe0-babb4318adc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:24:04.141112 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-04c1f070-e3a5-4a05-8fe0-babb4318adc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:24:04.150293 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1f070-e3a5-4a05-8fe0-babb4318adc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:24:04.151754 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1f070-e3a5-4a05-8fe0-babb4318adc9 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:24:04.159913 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1404/2773] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:24:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:24:08.276694 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef0b79-26b6-4cf7-baeb-cded20a08ea5 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:24:08.278170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f2ef0b79-26b6-4cf7-baeb-cded20a08ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:24:08.290280 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef0b79-26b6-4cf7-baeb-cded20a08ea5 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:24:08.294374 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e46697-aa0f-41e1-85cd-38f7b1d80650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:24:08.295062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e46697-aa0f-41e1-85cd-38f7b1d80650 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:24:08.295364 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e46697-aa0f-41e1-85cd-38f7b1d80650 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:24:08.295642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e46697-aa0f-41e1-85cd-38f7b1d80650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:24:08.297402 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-99e46697-aa0f-41e1-85cd-38f7b1d80650 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:24:08.307771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e46697-aa0f-41e1-85cd-38f7b1d80650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:24:08.309235 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e46697-aa0f-41e1-85cd-38f7b1d80650 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:24:08.317597 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1370/2774] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:24:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:24:08.391226 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1f070-e3a5-4a05-8fe0-babb4318adc9 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:24:08.400402 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1f070-e3a5-4a05-8fe0-babb4318adc9 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:24:08.404547 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e2d62-7a87-4f8b-b9de-ff1d4ecbc85d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:24:08.405228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e2d62-7a87-4f8b-b9de-ff1d4ecbc85d service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:24:08.405523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e2d62-7a87-4f8b-b9de-ff1d4ecbc85d service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:24:08.405929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e2d62-7a87-4f8b-b9de-ff1d4ecbc85d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:24:08.407730 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ff3e2d62-7a87-4f8b-b9de-ff1d4ecbc85d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:24:08.422160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e2d62-7a87-4f8b-b9de-ff1d4ecbc85d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:24:08.423685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e2d62-7a87-4f8b-b9de-ff1d4ecbc85d service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:24:08.436228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1405/2775] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:24:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:24:18.313817 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=6 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:24:18.430672 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=12 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:25:15.259986 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e46697-aa0f-41e1-85cd-38f7b1d80650 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:25:15.261368 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-99e46697-aa0f-41e1-85cd-38f7b1d80650 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:25:15.276931 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e46697-aa0f-41e1-85cd-38f7b1d80650 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:25:15.283387 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133851ea-6454-47f8-9b8d-f5d6ec5702f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:25:15.284020 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133851ea-6454-47f8-9b8d-f5d6ec5702f4 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:25:15.284304 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133851ea-6454-47f8-9b8d-f5d6ec5702f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:25:15.284582 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133851ea-6454-47f8-9b8d-f5d6ec5702f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:25:15.286279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-133851ea-6454-47f8-9b8d-f5d6ec5702f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:25:15.297467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133851ea-6454-47f8-9b8d-f5d6ec5702f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:25:15.299066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133851ea-6454-47f8-9b8d-f5d6ec5702f4 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:25:15.307951 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1371/2776] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:25:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:25:25.303888 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=6 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:25:33.509920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e2d62-7a87-4f8b-b9de-ff1d4ecbc85d service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:25:33.515043 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ff3e2d62-7a87-4f8b-b9de-ff1d4ecbc85d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:25:33.525544 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e2d62-7a87-4f8b-b9de-ff1d4ecbc85d service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:25:33.531425 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef3c8e-9805-4679-aa16-3f05c1141da8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:25:33.532079 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef3c8e-9805-4679-aa16-3f05c1141da8 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:25:33.532369 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef3c8e-9805-4679-aa16-3f05c1141da8 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:25:33.532651 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef3c8e-9805-4679-aa16-3f05c1141da8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:25:33.537750 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-83ef3c8e-9805-4679-aa16-3f05c1141da8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:25:33.547427 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef3c8e-9805-4679-aa16-3f05c1141da8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:25:33.548999 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef3c8e-9805-4679-aa16-3f05c1141da8 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:25:33.559961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1406/2777] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:25:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:25:37.950030 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6599b85-5841-4246-a626-5374c0d33d82 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:25:37.950553 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6599b85-5841-4246-a626-5374c0d33d82 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:25:37.950665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6599b85-5841-4246-a626-5374c0d33d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:25:37.968246 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d6599b85-5841-4246-a626-5374c0d33d82 None None] MFA Rules not processed for user `d49c23c64a684a39912c5cdfbeb1a1d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:25:37.969431 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d6599b85-5841-4246-a626-5374c0d33d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:25:37.976269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1372/2778] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:25:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:25:38.272712 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef3c8e-9805-4679-aa16-3f05c1141da8 service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:25:38.282592 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef3c8e-9805-4679-aa16-3f05c1141da8 service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:25:38.286628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2157bc-d0c0-4419-a1a7-0f11859352de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:25:38.287289 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2157bc-d0c0-4419-a1a7-0f11859352de service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:25:38.287577 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2157bc-d0c0-4419-a1a7-0f11859352de service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:25:38.287852 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2157bc-d0c0-4419-a1a7-0f11859352de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:25:38.289625 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ea2157bc-d0c0-4419-a1a7-0f11859352de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:25:38.298872 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2157bc-d0c0-4419-a1a7-0f11859352de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:25:38.300964 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2157bc-d0c0-4419-a1a7-0f11859352de service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:25:38.308832 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1407/2779] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:25:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:25:38.706388 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6599b85-5841-4246-a626-5374c0d33d82 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:25:38.707992 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d6599b85-5841-4246-a626-5374c0d33d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:25:38.778784 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6599b85-5841-4246-a626-5374c0d33d82 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:25:38.784643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a9c645-1c65-496c-a82c-4ae16d127d61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c577fa8670e34449a80de0c8e0a15a8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:25:38.785287 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9c645-1c65-496c-a82c-4ae16d127d61 service placement] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:25:38.785578 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9c645-1c65-496c-a82c-4ae16d127d61 service placement] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:25:38.785857 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9c645-1c65-496c-a82c-4ae16d127d61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:25:38.787645 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c9a9c645-1c65-496c-a82c-4ae16d127d61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:25:38.798841 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a9c645-1c65-496c-a82c-4ae16d127d61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:25:38.800488 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a9c645-1c65-496c-a82c-4ae16d127d61 service placement] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:25:38.810288 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1373/2780] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:25:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:25:48.305258 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=12 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:25:48.806077 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=35 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:25:50.089350 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2157bc-d0c0-4419-a1a7-0f11859352de service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:25:50.089350 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2157bc-d0c0-4419-a1a7-0f11859352de service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:25:50.089350 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd3165-392c-4593-9794-b690996e95b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:25:50.089350 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd3165-392c-4593-9794-b690996e95b0 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:25:50.090250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd3165-392c-4593-9794-b690996e95b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:25:50.090250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd3165-392c-4593-9794-b690996e95b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:25:50.090250 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-81cd3165-392c-4593-9794-b690996e95b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:25:50.095668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd3165-392c-4593-9794-b690996e95b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:25:50.097242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd3165-392c-4593-9794-b690996e95b0 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:25:50.107702 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1408/2781] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:25:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:26:00.103227 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=6 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:26:04.699919 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a9c645-1c65-496c-a82c-4ae16d127d61 service placement] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:26:04.709752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a9c645-1c65-496c-a82c-4ae16d127d61 service placement] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:26:04.713315 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0549e5-51da-467e-aff2-cb10e0f12de6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:26:04.713943 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0549e5-51da-467e-aff2-cb10e0f12de6 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:26:04.714229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0549e5-51da-467e-aff2-cb10e0f12de6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:26:04.714506 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0549e5-51da-467e-aff2-cb10e0f12de6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:26:04.716258 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9d0549e5-51da-467e-aff2-cb10e0f12de6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:26:04.783856 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0549e5-51da-467e-aff2-cb10e0f12de6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:26:04.785208 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0549e5-51da-467e-aff2-cb10e0f12de6 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:26:04.794021 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1374/2782] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:26:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:26:05.682945 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd3165-392c-4593-9794-b690996e95b0 service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:26:05.735684 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd3165-392c-4593-9794-b690996e95b0 service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:26:05.741741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0309d6a-f407-4f6f-b412-9c0157f8b144 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d27d6df5cb64697adebd961dfbec324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:26:05.742463 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0309d6a-f407-4f6f-b412-9c0157f8b144 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:26:05.742761 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0309d6a-f407-4f6f-b412-9c0157f8b144 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:26:05.743095 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0309d6a-f407-4f6f-b412-9c0157f8b144 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:26:05.745489 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b0309d6a-f407-4f6f-b412-9c0157f8b144 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:26:05.769121 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0309d6a-f407-4f6f-b412-9c0157f8b144 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:26:05.770740 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0309d6a-f407-4f6f-b412-9c0157f8b144 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:26:05.782724 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1409/2783] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:26:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:26:11.012493 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0549e5-51da-467e-aff2-cb10e0f12de6 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:26:11.042237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0549e5-51da-467e-aff2-cb10e0f12de6 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:26:11.045824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af5297-edb4-49bb-8b5f-40064eb35840 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:26:11.046463 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af5297-edb4-49bb-8b5f-40064eb35840 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:26:11.046746 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af5297-edb4-49bb-8b5f-40064eb35840 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:26:11.047023 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af5297-edb4-49bb-8b5f-40064eb35840 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:26:11.048810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-03af5297-edb4-49bb-8b5f-40064eb35840 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:26:11.133282 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af5297-edb4-49bb-8b5f-40064eb35840 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:26:11.134723 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af5297-edb4-49bb-8b5f-40064eb35840 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:26:11.156923 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1375/2784] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:26:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:26:15.777650 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=24 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:26:17.954355 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0309d6a-f407-4f6f-b412-9c0157f8b144 service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:26:17.975553 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0309d6a-f407-4f6f-b412-9c0157f8b144 service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:26:17.980579 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa6c26-6f76-4bbf-993c-014acb76775c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c577fa8670e34449a80de0c8e0a15a8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:26:17.981224 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa6c26-6f76-4bbf-993c-014acb76775c service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:26:17.981508 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa6c26-6f76-4bbf-993c-014acb76775c service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:26:17.981786 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa6c26-6f76-4bbf-993c-014acb76775c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:26:17.983576 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a6fa6c26-6f76-4bbf-993c-014acb76775c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:26:18.015748 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa6c26-6f76-4bbf-993c-014acb76775c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:26:18.020648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa6c26-6f76-4bbf-993c-014acb76775c service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:26:18.027305 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1410/2785] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:26:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:26:18.076823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af5297-edb4-49bb-8b5f-40064eb35840 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:26:18.078288 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-03af5297-edb4-49bb-8b5f-40064eb35840 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:26:18.090840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af5297-edb4-49bb-8b5f-40064eb35840 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:26:18.096911 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915df559-21d4-40c8-a392-ba510917ac50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:26:18.097724 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915df559-21d4-40c8-a392-ba510917ac50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:26:18.098004 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915df559-21d4-40c8-a392-ba510917ac50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:26:18.098277 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915df559-21d4-40c8-a392-ba510917ac50 admin admin] PATH_INFO: `/v3/users/a38c1207b03d4125bcbe99c569573dc6` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:26:18.100102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915df559-21d4-40c8-a392-ba510917ac50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a38c1207b03d4125bcbe99c569573dc6)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:26:18.102018 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915df559-21d4-40c8-a392-ba510917ac50 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:26:18.264986 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:26:18.278092 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-915df559-21d4-40c8-a392-ba510917ac50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a38c1207b03d4125bcbe99c569573dc6'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:18.285123 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-915df559-21d4-40c8-a392-ba510917ac50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a38c1207b03d4125bcbe99c569573dc6'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:18.285829 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-915df559-21d4-40c8-a392-ba510917ac50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a38c1207b03d4125bcbe99c569573dc6'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:18.286979 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-915df559-21d4-40c8-a392-ba510917ac50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a38c1207b03d4125bcbe99c569573dc6'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:18.298354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-915df559-21d4-40c8-a392-ba510917ac50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a38c1207b03d4125bcbe99c569573dc6'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:18.372038 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1376/2786] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:26:18 2022] DELETE /identity/v3/users/a38c1207b03d4125bcbe99c569573dc6 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:26:18.628628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa6c26-6f76-4bbf-993c-014acb76775c service placement] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:26:18.639734 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa6c26-6f76-4bbf-993c-014acb76775c service placement] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:26:18.698936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14f4e3-3b4a-4114-bf10-338e9eabc8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:26:18.699829 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14f4e3-3b4a-4114-bf10-338e9eabc8b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:26:18.700121 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14f4e3-3b4a-4114-bf10-338e9eabc8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:26:18.700411 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14f4e3-3b4a-4114-bf10-338e9eabc8b8 admin admin] PATH_INFO: `/v3/projects/97d0d5140e26495f856abe6dc4ce8549` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:26:18.701561 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14f4e3-3b4a-4114-bf10-338e9eabc8b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97d0d5140e26495f856abe6dc4ce8549)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:26:18.703525 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14f4e3-3b4a-4114-bf10-338e9eabc8b8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:26:18.807905 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7a14f4e3-3b4a-4114-bf10-338e9eabc8b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97d0d5140e26495f856abe6dc4ce8549', 'request_id': 'req-7a14f4e3-3b4a-4114-bf10-338e9eabc8b8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:18.869855 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7a14f4e3-3b4a-4114-bf10-338e9eabc8b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97d0d5140e26495f856abe6dc4ce8549', 'request_id': 'req-7a14f4e3-3b4a-4114-bf10-338e9eabc8b8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:18.875623 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7a14f4e3-3b4a-4114-bf10-338e9eabc8b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97d0d5140e26495f856abe6dc4ce8549', 'request_id': 'req-7a14f4e3-3b4a-4114-bf10-338e9eabc8b8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:18.886571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7a14f4e3-3b4a-4114-bf10-338e9eabc8b8 admin admin] The token cache is being invalidate because project 97d0d5140e26495f856abe6dc4ce8549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:26:18.886858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7a14f4e3-3b4a-4114-bf10-338e9eabc8b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:18.887502 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7a14f4e3-3b4a-4114-bf10-338e9eabc8b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:18.889034 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1411/2787] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:26:18 2022] DELETE /identity/v3/projects/97d0d5140e26495f856abe6dc4ce8549 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:26:18.893123 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915df559-21d4-40c8-a392-ba510917ac50 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:26:18.894851 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-915df559-21d4-40c8-a392-ba510917ac50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:26:18.908505 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915df559-21d4-40c8-a392-ba510917ac50 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:26:18.963989 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f9ee2-18e9-489e-9bac-a52e1c29940a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:26:18.964866 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f9ee2-18e9-489e-9bac-a52e1c29940a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:26:18.965156 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f9ee2-18e9-489e-9bac-a52e1c29940a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:26:18.965440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f9ee2-18e9-489e-9bac-a52e1c29940a admin admin] PATH_INFO: `/v3/users/6d83b42d8333430d9cc00a3afedf9304` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:26:18.966674 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f9ee2-18e9-489e-9bac-a52e1c29940a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d83b42d8333430d9cc00a3afedf9304)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:26:18.968635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f9ee2-18e9-489e-9bac-a52e1c29940a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:26:19.107392 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6e6f9ee2-18e9-489e-9bac-a52e1c29940a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d83b42d8333430d9cc00a3afedf9304'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:19.114363 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6e6f9ee2-18e9-489e-9bac-a52e1c29940a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d83b42d8333430d9cc00a3afedf9304'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:19.114974 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6e6f9ee2-18e9-489e-9bac-a52e1c29940a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d83b42d8333430d9cc00a3afedf9304'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:19.115677 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6e6f9ee2-18e9-489e-9bac-a52e1c29940a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d83b42d8333430d9cc00a3afedf9304'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:19.127704 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6e6f9ee2-18e9-489e-9bac-a52e1c29940a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d83b42d8333430d9cc00a3afedf9304'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:19.201066 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1377/2788] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:26:18 2022] DELETE /identity/v3/users/6d83b42d8333430d9cc00a3afedf9304 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:26:19.443236 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14f4e3-3b4a-4114-bf10-338e9eabc8b8 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:26:19.446801 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7a14f4e3-3b4a-4114-bf10-338e9eabc8b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:26:19.457835 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14f4e3-3b4a-4114-bf10-338e9eabc8b8 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:26:19.515075 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:26:19.521820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913df2a-94e0-4ebc-be88-5a20ecf09a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:26:19.522805 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913df2a-94e0-4ebc-be88-5a20ecf09a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:26:19.523094 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913df2a-94e0-4ebc-be88-5a20ecf09a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:26:19.523551 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913df2a-94e0-4ebc-be88-5a20ecf09a3f admin admin] PATH_INFO: `/v3/projects/d9180ad54c1e4532a67d99621f6363b2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:26:19.524710 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913df2a-94e0-4ebc-be88-5a20ecf09a3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9180ad54c1e4532a67d99621f6363b2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:26:19.526659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913df2a-94e0-4ebc-be88-5a20ecf09a3f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:26:19.667455 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c913df2a-94e0-4ebc-be88-5a20ecf09a3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9180ad54c1e4532a67d99621f6363b2', 'request_id': 'req-c913df2a-94e0-4ebc-be88-5a20ecf09a3f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:19.733364 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c913df2a-94e0-4ebc-be88-5a20ecf09a3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9180ad54c1e4532a67d99621f6363b2', 'request_id': 'req-c913df2a-94e0-4ebc-be88-5a20ecf09a3f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:19.740026 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c913df2a-94e0-4ebc-be88-5a20ecf09a3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9180ad54c1e4532a67d99621f6363b2', 'request_id': 'req-c913df2a-94e0-4ebc-be88-5a20ecf09a3f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:19.751122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c913df2a-94e0-4ebc-be88-5a20ecf09a3f admin admin] The token cache is being invalidate because project d9180ad54c1e4532a67d99621f6363b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:26:19.751463 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c913df2a-94e0-4ebc-be88-5a20ecf09a3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:19.752110 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c913df2a-94e0-4ebc-be88-5a20ecf09a3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:26:19.753851 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1412/2789] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:26:19 2022] DELETE /identity/v3/projects/d9180ad54c1e4532a67d99621f6363b2 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:26:29.197353 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=71,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:26:29.748086 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:05.077503 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f9ee2-18e9-489e-9bac-a52e1c29940a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:05.078896 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6e6f9ee2-18e9-489e-9bac-a52e1c29940a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:05.147992 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f9ee2-18e9-489e-9bac-a52e1c29940a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:05.208666 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e9c46-4ece-4eec-8d6b-3147e1c0e700 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c577fa8670e34449a80de0c8e0a15a8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:05.209320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e9c46-4ece-4eec-8d6b-3147e1c0e700 service placement] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:05.209610 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e9c46-4ece-4eec-8d6b-3147e1c0e700 service placement] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:05.209888 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e9c46-4ece-4eec-8d6b-3147e1c0e700 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:05.211682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3e8e9c46-4ece-4eec-8d6b-3147e1c0e700 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:05.223261 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e9c46-4ece-4eec-8d6b-3147e1c0e700 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:05.225434 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e9c46-4ece-4eec-8d6b-3147e1c0e700 service placement] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:05.235584 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1378/2790] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:27:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:27:15.231093 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=58 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:20.014430 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913df2a-94e0-4ebc-be88-5a20ecf09a3f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:20.015888 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c913df2a-94e0-4ebc-be88-5a20ecf09a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:20.084267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913df2a-94e0-4ebc-be88-5a20ecf09a3f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:20.088000 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75112e56-32b5-405c-89e0-888136c5c673 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:20.088680 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75112e56-32b5-405c-89e0-888136c5c673 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:20.088968 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75112e56-32b5-405c-89e0-888136c5c673 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:20.089241 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75112e56-32b5-405c-89e0-888136c5c673 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:20.091021 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-75112e56-32b5-405c-89e0-888136c5c673 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:20.151174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75112e56-32b5-405c-89e0-888136c5c673 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:20.152630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75112e56-32b5-405c-89e0-888136c5c673 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:20.162206 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1413/2791] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:27:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:27:23.729596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e9c46-4ece-4eec-8d6b-3147e1c0e700 service placement] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:23.731053 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3e8e9c46-4ece-4eec-8d6b-3147e1c0e700 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:23.746199 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e9c46-4ece-4eec-8d6b-3147e1c0e700 service placement] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:23.750845 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d3964-1085-45b8-aa26-b8ff026fbe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:23.751766 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d3964-1085-45b8-aa26-b8ff026fbe0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:23.752056 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d3964-1085-45b8-aa26-b8ff026fbe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:23.752342 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d3964-1085-45b8-aa26-b8ff026fbe0c admin admin] PATH_INFO: `/v3/users/d49c23c64a684a39912c5cdfbeb1a1d5` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:23.753542 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d3964-1085-45b8-aa26-b8ff026fbe0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d49c23c64a684a39912c5cdfbeb1a1d5)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:23.755617 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d3964-1085-45b8-aa26-b8ff026fbe0c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:23.896268 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9b8d3964-1085-45b8-aa26-b8ff026fbe0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd49c23c64a684a39912c5cdfbeb1a1d5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:23.903133 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9b8d3964-1085-45b8-aa26-b8ff026fbe0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd49c23c64a684a39912c5cdfbeb1a1d5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:23.903830 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9b8d3964-1085-45b8-aa26-b8ff026fbe0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd49c23c64a684a39912c5cdfbeb1a1d5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:23.904462 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9b8d3964-1085-45b8-aa26-b8ff026fbe0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd49c23c64a684a39912c5cdfbeb1a1d5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:23.916821 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9b8d3964-1085-45b8-aa26-b8ff026fbe0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd49c23c64a684a39912c5cdfbeb1a1d5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:24.004886 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1379/2792] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:27:23 2022] DELETE /identity/v3/users/d49c23c64a684a39912c5cdfbeb1a1d5 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:24.247638 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75112e56-32b5-405c-89e0-888136c5c673 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:24.254993 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75112e56-32b5-405c-89e0-888136c5c673 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:24.315064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5149e-f54a-4342-adc1-b991148f9a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:24.315987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5149e-f54a-4342-adc1-b991148f9a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:24.316282 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5149e-f54a-4342-adc1-b991148f9a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:24.316563 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5149e-f54a-4342-adc1-b991148f9a54 admin admin] PATH_INFO: `/v3/projects/2f0e7f15120d41fba8d239623faee2ec` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:24.317840 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5149e-f54a-4342-adc1-b991148f9a54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f0e7f15120d41fba8d239623faee2ec)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:24.319922 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5149e-f54a-4342-adc1-b991148f9a54 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:24.428839 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:24.434784 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0de5149e-f54a-4342-adc1-b991148f9a54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f0e7f15120d41fba8d239623faee2ec', 'request_id': 'req-0de5149e-f54a-4342-adc1-b991148f9a54'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:24.511337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0de5149e-f54a-4342-adc1-b991148f9a54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f0e7f15120d41fba8d239623faee2ec', 'request_id': 'req-0de5149e-f54a-4342-adc1-b991148f9a54'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:24.516913 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0de5149e-f54a-4342-adc1-b991148f9a54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f0e7f15120d41fba8d239623faee2ec', 'request_id': 'req-0de5149e-f54a-4342-adc1-b991148f9a54'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:24.529722 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0de5149e-f54a-4342-adc1-b991148f9a54 admin admin] The token cache is being invalidate because project 2f0e7f15120d41fba8d239623faee2ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:27:24.530259 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0de5149e-f54a-4342-adc1-b991148f9a54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:24.530898 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0de5149e-f54a-4342-adc1-b991148f9a54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:24.532451 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1414/2793] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:27:24 2022] DELETE /identity/v3/projects/2f0e7f15120d41fba8d239623faee2ec => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:24.537479 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d80d3-a6d8-4153-8641-add566ab59d7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:24.537807 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d80d3-a6d8-4153-8641-add566ab59d7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:24.538121 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d80d3-a6d8-4153-8641-add566ab59d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:24.595719 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-5d2d80d3-a6d8-4153-8641-add566ab59d7 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:24.596970 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5d2d80d3-a6d8-4153-8641-add566ab59d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:24.657973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1380/2794] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:27:24.663675 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354be02-e349-41fd-9336-ffe6661736cd None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:24.663877 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354be02-e349-41fd-9336-ffe6661736cd None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:24.664160 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354be02-e349-41fd-9336-ffe6661736cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:24.683462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7354be02-e349-41fd-9336-ffe6661736cd None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:24.684689 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7354be02-e349-41fd-9336-ffe6661736cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:24.692523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1415/2795] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:27:24.697196 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d80d3-a6d8-4153-8641-add566ab59d7 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:24.706605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d80d3-a6d8-4153-8641-add566ab59d7 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:24.711777 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d24a82-f92e-4602-b4c0-78154c8ffa93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:24.712424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d24a82-f92e-4602-b4c0-78154c8ffa93 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:24.712708 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d24a82-f92e-4602-b4c0-78154c8ffa93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:24.712982 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d24a82-f92e-4602-b4c0-78154c8ffa93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:24.713697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d24a82-f92e-4602-b4c0-78154c8ffa93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:27:24.714004 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d24a82-f92e-4602-b4c0-78154c8ffa93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:24.715633 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d24a82-f92e-4602-b4c0-78154c8ffa93 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:24.726980 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1381/2796] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:27:24.731291 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354be02-e349-41fd-9336-ffe6661736cd None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:24.740495 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354be02-e349-41fd-9336-ffe6661736cd None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:24.745119 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f432bd2-dff0-4bf0-92c4-9b850f8e4c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:24.745764 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f432bd2-dff0-4bf0-92c4-9b850f8e4c1e admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:24.746050 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f432bd2-dff0-4bf0-92c4-9b850f8e4c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:24.746330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f432bd2-dff0-4bf0-92c4-9b850f8e4c1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:24.747027 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f432bd2-dff0-4bf0-92c4-9b850f8e4c1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:24.748955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f432bd2-dff0-4bf0-92c4-9b850f8e4c1e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:24.822550 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1416/2797] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:27:24 2022] POST /identity/v3/projects => generated 393 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:27:24.826671 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d24a82-f92e-4602-b4c0-78154c8ffa93 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:24.837422 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d24a82-f92e-4602-b4c0-78154c8ffa93 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:24.864075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:24.894245 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941820b-fba3-42fe-9291-6e00a87adfc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:24.898753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941820b-fba3-42fe-9291-6e00a87adfc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:24.898753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941820b-fba3-42fe-9291-6e00a87adfc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:24.898753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941820b-fba3-42fe-9291-6e00a87adfc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:24.898753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941820b-fba3-42fe-9291-6e00a87adfc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:24.898753 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941820b-fba3-42fe-9291-6e00a87adfc4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:25.018186 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1382/2798] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:27:24 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:27:25.022154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f432bd2-dff0-4bf0-92c4-9b850f8e4c1e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:25.032623 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f432bd2-dff0-4bf0-92c4-9b850f8e4c1e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:25.037174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce29142-b7db-4ea1-80b5-4efd47ee22ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:25.037830 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce29142-b7db-4ea1-80b5-4efd47ee22ac admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:25.038117 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce29142-b7db-4ea1-80b5-4efd47ee22ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:25.038396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce29142-b7db-4ea1-80b5-4efd47ee22ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:25.039116 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce29142-b7db-4ea1-80b5-4efd47ee22ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:25.040796 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce29142-b7db-4ea1-80b5-4efd47ee22ac admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:25.051003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1417/2799] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:27:25.055413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941820b-fba3-42fe-9291-6e00a87adfc4 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:25.065335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941820b-fba3-42fe-9291-6e00a87adfc4 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:25.070033 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5928c3-1a52-4022-84f2-2f38cb4f0efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:25.071189 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5928c3-1a52-4022-84f2-2f38cb4f0efa admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:25.071496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5928c3-1a52-4022-84f2-2f38cb4f0efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:25.071772 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5928c3-1a52-4022-84f2-2f38cb4f0efa admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:25.072486 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5928c3-1a52-4022-84f2-2f38cb4f0efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:25.074059 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5928c3-1a52-4022-84f2-2f38cb4f0efa admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:25.083999 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1383/2800] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:27:25.088059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce29142-b7db-4ea1-80b5-4efd47ee22ac admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:25.097850 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce29142-b7db-4ea1-80b5-4efd47ee22ac admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:25.102328 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e1ff2-e0d5-4f37-b32d-cf636d70062b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:25.103218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e1ff2-e0d5-4f37-b32d-cf636d70062b admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:25.103511 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e1ff2-e0d5-4f37-b32d-cf636d70062b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:25.103792 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e1ff2-e0d5-4f37-b32d-cf636d70062b admin admin] PATH_INFO: `/v3/projects/81b88a26d6964df5aa1ffee45e81ce63/users/5037423e822643afa108008d9afb8e35/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:25.118322 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e1ff2-e0d5-4f37-b32d-cf636d70062b admin admin] RBAC: Authorizing `identity:create_grant(project_id=81b88a26d6964df5aa1ffee45e81ce63, user_id=5037423e822643afa108008d9afb8e35, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:25.120976 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e1ff2-e0d5-4f37-b32d-cf636d70062b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:25.186443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1418/2801] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:27:25 2022] PUT /identity/v3/projects/81b88a26d6964df5aa1ffee45e81ce63/users/5037423e822643afa108008d9afb8e35/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:25.198073 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6656cc0-b4ea-49c8-adcb-f84f8c85d46e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:25.198465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6656cc0-b4ea-49c8-adcb-f84f8c85d46e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:25.198746 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6656cc0-b4ea-49c8-adcb-f84f8c85d46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:25.224537 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d6656cc0-b4ea-49c8-adcb-f84f8c85d46e None None] MFA Rules not processed for user `5037423e822643afa108008d9afb8e35`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:25.225736 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d6656cc0-b4ea-49c8-adcb-f84f8c85d46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:25.289707 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1384/2802] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:27:25.294278 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e1ff2-e0d5-4f37-b32d-cf636d70062b admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:25.355072 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e1ff2-e0d5-4f37-b32d-cf636d70062b admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:25.359726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d1a28-e3f3-4bc1-9e41-01c2686db44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:25.360387 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d1a28-e3f3-4bc1-9e41-01c2686db44c admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:25.360678 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d1a28-e3f3-4bc1-9e41-01c2686db44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:25.360961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d1a28-e3f3-4bc1-9e41-01c2686db44c admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:25.361704 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d1a28-e3f3-4bc1-9e41-01c2686db44c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:25.363639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d1a28-e3f3-4bc1-9e41-01c2686db44c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:25.442358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1419/2803] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:27:25 2022] POST /identity/v3/projects => generated 393 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:27:25.447068 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6656cc0-b4ea-49c8-adcb-f84f8c85d46e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:25.457535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6656cc0-b4ea-49c8-adcb-f84f8c85d46e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:25.512560 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ec928-5861-4e10-9667-68a42ed5d947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:25.513221 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ec928-5861-4e10-9667-68a42ed5d947 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:25.513502 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ec928-5861-4e10-9667-68a42ed5d947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:25.513784 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ec928-5861-4e10-9667-68a42ed5d947 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:25.514500 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ec928-5861-4e10-9667-68a42ed5d947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:25.516465 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ec928-5861-4e10-9667-68a42ed5d947 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:25.646963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1385/2804] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:27:25 2022] POST /identity/v3/users => generated 303 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:27:25.651123 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d1a28-e3f3-4bc1-9e41-01c2686db44c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:25.661575 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d1a28-e3f3-4bc1-9e41-01c2686db44c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:25.666067 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba88e9-5f15-42f0-bd7e-b12f2ae3e60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:25.666718 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba88e9-5f15-42f0-bd7e-b12f2ae3e60b admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:25.667002 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba88e9-5f15-42f0-bd7e-b12f2ae3e60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:25.667313 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba88e9-5f15-42f0-bd7e-b12f2ae3e60b admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:25.668131 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba88e9-5f15-42f0-bd7e-b12f2ae3e60b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:25.669727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba88e9-5f15-42f0-bd7e-b12f2ae3e60b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:25.680047 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1420/2805] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:27:25.684544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ec928-5861-4e10-9667-68a42ed5d947 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:25.694615 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ec928-5861-4e10-9667-68a42ed5d947 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:25.700011 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d3fbf-0a96-4759-ad6b-ec9c4f6b36de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:25.700880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d3fbf-0a96-4759-ad6b-ec9c4f6b36de admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:25.701164 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d3fbf-0a96-4759-ad6b-ec9c4f6b36de admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:25.701438 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d3fbf-0a96-4759-ad6b-ec9c4f6b36de admin admin] PATH_INFO: `/v3/projects/d9f16982615f4dbf8ed46a3ec038f65e/users/90b16e382eb04746ba775a9372dd8149/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:25.716044 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d3fbf-0a96-4759-ad6b-ec9c4f6b36de admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f16982615f4dbf8ed46a3ec038f65e, user_id=90b16e382eb04746ba775a9372dd8149, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:25.718728 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d3fbf-0a96-4759-ad6b-ec9c4f6b36de admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:25.824145 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1386/2806] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:27:25 2022] PUT /identity/v3/projects/d9f16982615f4dbf8ed46a3ec038f65e/users/90b16e382eb04746ba775a9372dd8149/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:25.828162 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba88e9-5f15-42f0-bd7e-b12f2ae3e60b admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:25.837957 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba88e9-5f15-42f0-bd7e-b12f2ae3e60b admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:25.875114 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:25.893015 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b645f-7786-4537-949a-9d34b4bd987c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:25.893676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b645f-7786-4537-949a-9d34b4bd987c admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:25.893968 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b645f-7786-4537-949a-9d34b4bd987c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:25.894249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b645f-7786-4537-949a-9d34b4bd987c admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:25.895000 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b645f-7786-4537-949a-9d34b4bd987c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:25.896646 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b645f-7786-4537-949a-9d34b4bd987c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:25.906493 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1421/2807] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:27:25.910739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d3fbf-0a96-4759-ad6b-ec9c4f6b36de admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:25.916186 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:25.921358 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d3fbf-0a96-4759-ad6b-ec9c4f6b36de admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:25.925915 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ae0cc-346e-430f-921b-7079cbcac88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:25.926789 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ae0cc-346e-430f-921b-7079cbcac88b admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:25.927079 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ae0cc-346e-430f-921b-7079cbcac88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:25.927383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ae0cc-346e-430f-921b-7079cbcac88b admin admin] PATH_INFO: `/v3/projects/d9f16982615f4dbf8ed46a3ec038f65e/users/90b16e382eb04746ba775a9372dd8149/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:25.929325 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ae0cc-346e-430f-921b-7079cbcac88b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f16982615f4dbf8ed46a3ec038f65e, user_id=90b16e382eb04746ba775a9372dd8149, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:25.932549 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ae0cc-346e-430f-921b-7079cbcac88b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:25.942825 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1387/2808] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:27:25 2022] PUT /identity/v3/projects/d9f16982615f4dbf8ed46a3ec038f65e/users/90b16e382eb04746ba775a9372dd8149/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:25.953325 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805bd9a-e0d5-404e-948d-4a21a62512a9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:25.953620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805bd9a-e0d5-404e-948d-4a21a62512a9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:25.953906 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805bd9a-e0d5-404e-948d-4a21a62512a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:25.980682 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5805bd9a-e0d5-404e-948d-4a21a62512a9 None None] MFA Rules not processed for user `90b16e382eb04746ba775a9372dd8149`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:25.981912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5805bd9a-e0d5-404e-948d-4a21a62512a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:26.056245 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1422/2809] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:27:26.066780 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ae0cc-346e-430f-921b-7079cbcac88b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:26.068599 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-667ae0cc-346e-430f-921b-7079cbcac88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:26.145618 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ae0cc-346e-430f-921b-7079cbcac88b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:26.149311 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91cd7d6-b616-44d5-8eb1-b0d8198515fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:26.150044 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91cd7d6-b616-44d5-8eb1-b0d8198515fb service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:26.150333 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91cd7d6-b616-44d5-8eb1-b0d8198515fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:26.150608 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91cd7d6-b616-44d5-8eb1-b0d8198515fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:26.159776 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91cd7d6-b616-44d5-8eb1-b0d8198515fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:26.161148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91cd7d6-b616-44d5-8eb1-b0d8198515fb service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:26.170788 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1388/2810] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:27:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:27:26.215118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805bd9a-e0d5-404e-948d-4a21a62512a9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:26.216548 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5805bd9a-e0d5-404e-948d-4a21a62512a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:26.228852 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805bd9a-e0d5-404e-948d-4a21a62512a9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:26.232682 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90187c12-86d6-47a4-8a55-e162f1131bcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:26.233322 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90187c12-86d6-47a4-8a55-e162f1131bcd service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:26.233615 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90187c12-86d6-47a4-8a55-e162f1131bcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:26.233899 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90187c12-86d6-47a4-8a55-e162f1131bcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:26.288986 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90187c12-86d6-47a4-8a55-e162f1131bcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:26.291004 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90187c12-86d6-47a4-8a55-e162f1131bcd service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:26.299845 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1423/2811] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:27:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:27:30.786970 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91cd7d6-b616-44d5-8eb1-b0d8198515fb service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:30.848880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91cd7d6-b616-44d5-8eb1-b0d8198515fb service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:30.853951 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca9065-83e6-4c39-8d2c-5420b0b57cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:30.854836 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca9065-83e6-4c39-8d2c-5420b0b57cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:30.855120 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca9065-83e6-4c39-8d2c-5420b0b57cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:30.855433 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca9065-83e6-4c39-8d2c-5420b0b57cd1 admin admin] PATH_INFO: `/v3/users/5037423e822643afa108008d9afb8e35` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:30.856596 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca9065-83e6-4c39-8d2c-5420b0b57cd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5037423e822643afa108008d9afb8e35)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:30.858492 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca9065-83e6-4c39-8d2c-5420b0b57cd1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:31.003348 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-54ca9065-83e6-4c39-8d2c-5420b0b57cd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5037423e822643afa108008d9afb8e35'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:31.010513 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-54ca9065-83e6-4c39-8d2c-5420b0b57cd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5037423e822643afa108008d9afb8e35'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:31.011225 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-54ca9065-83e6-4c39-8d2c-5420b0b57cd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5037423e822643afa108008d9afb8e35'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:31.011856 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-54ca9065-83e6-4c39-8d2c-5420b0b57cd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5037423e822643afa108008d9afb8e35'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:31.023643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-54ca9065-83e6-4c39-8d2c-5420b0b57cd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5037423e822643afa108008d9afb8e35'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:31.026043 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:31.085969 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1389/2812] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:27:30 2022] DELETE /identity/v3/users/5037423e822643afa108008d9afb8e35 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:31.091063 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2f903-4587-489f-9962-5b5713aa2971 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:31.091549 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2f903-4587-489f-9962-5b5713aa2971 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:31.096695 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2f903-4587-489f-9962-5b5713aa2971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:31.109412 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7bf2f903-4587-489f-9962-5b5713aa2971 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:31.110539 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7bf2f903-4587-489f-9962-5b5713aa2971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:31.159771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:31.168496 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1424/2813] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:27:31.178202 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca9065-83e6-4c39-8d2c-5420b0b57cd1 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:31.179634 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-54ca9065-83e6-4c39-8d2c-5420b0b57cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:31.235859 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca9065-83e6-4c39-8d2c-5420b0b57cd1 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:31.239408 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0972be5-ae23-4c81-af6d-038cc874d25a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:31.240043 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0972be5-ae23-4c81-af6d-038cc874d25a service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:31.240329 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0972be5-ae23-4c81-af6d-038cc874d25a service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:31.240607 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0972be5-ae23-4c81-af6d-038cc874d25a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:31.249429 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0972be5-ae23-4c81-af6d-038cc874d25a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:31.250847 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0972be5-ae23-4c81-af6d-038cc874d25a service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:31.260616 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1390/2814] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:27:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:27:31.650210 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2f903-4587-489f-9962-5b5713aa2971 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:31.651743 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7bf2f903-4587-489f-9962-5b5713aa2971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:31.664885 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2f903-4587-489f-9962-5b5713aa2971 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:31.669657 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b377b-590b-48b4-aa8e-676c2d4d52b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:31.670433 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b377b-590b-48b4-aa8e-676c2d4d52b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:31.670721 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b377b-590b-48b4-aa8e-676c2d4d52b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:31.671003 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b377b-590b-48b4-aa8e-676c2d4d52b4 admin admin] PATH_INFO: `/v3/projects/81b88a26d6964df5aa1ffee45e81ce63` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:31.672253 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b377b-590b-48b4-aa8e-676c2d4d52b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81b88a26d6964df5aa1ffee45e81ce63)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:31.674280 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b377b-590b-48b4-aa8e-676c2d4d52b4 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:31.767330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-110b377b-590b-48b4-aa8e-676c2d4d52b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81b88a26d6964df5aa1ffee45e81ce63', 'request_id': 'req-110b377b-590b-48b4-aa8e-676c2d4d52b4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:31.825739 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-110b377b-590b-48b4-aa8e-676c2d4d52b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81b88a26d6964df5aa1ffee45e81ce63', 'request_id': 'req-110b377b-590b-48b4-aa8e-676c2d4d52b4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:31.831490 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-110b377b-590b-48b4-aa8e-676c2d4d52b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81b88a26d6964df5aa1ffee45e81ce63', 'request_id': 'req-110b377b-590b-48b4-aa8e-676c2d4d52b4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:31.842476 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-110b377b-590b-48b4-aa8e-676c2d4d52b4 admin admin] The token cache is being invalidate because project 81b88a26d6964df5aa1ffee45e81ce63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:27:31.842761 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-110b377b-590b-48b4-aa8e-676c2d4d52b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:31.843430 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-110b377b-590b-48b4-aa8e-676c2d4d52b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:31.844955 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1425/2815] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:27:31 2022] DELETE /identity/v3/projects/81b88a26d6964df5aa1ffee45e81ce63 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:31.849059 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0972be5-ae23-4c81-af6d-038cc874d25a service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:31.858504 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0972be5-ae23-4c81-af6d-038cc874d25a service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:31.914913 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2776ae7d-859f-4d74-bc60-6264bfecc9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:31.915745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776ae7d-859f-4d74-bc60-6264bfecc9cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:31.916037 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776ae7d-859f-4d74-bc60-6264bfecc9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:31.916319 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776ae7d-859f-4d74-bc60-6264bfecc9cf admin admin] PATH_INFO: `/v3/users/90b16e382eb04746ba775a9372dd8149` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:31.917510 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2776ae7d-859f-4d74-bc60-6264bfecc9cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=90b16e382eb04746ba775a9372dd8149)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:31.919458 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2776ae7d-859f-4d74-bc60-6264bfecc9cf admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:32.102827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2776ae7d-859f-4d74-bc60-6264bfecc9cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90b16e382eb04746ba775a9372dd8149'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:32.110162 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2776ae7d-859f-4d74-bc60-6264bfecc9cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90b16e382eb04746ba775a9372dd8149'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:32.110855 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2776ae7d-859f-4d74-bc60-6264bfecc9cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90b16e382eb04746ba775a9372dd8149'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:32.111494 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2776ae7d-859f-4d74-bc60-6264bfecc9cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90b16e382eb04746ba775a9372dd8149'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:32.124151 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2776ae7d-859f-4d74-bc60-6264bfecc9cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90b16e382eb04746ba775a9372dd8149'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:32.191333 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1391/2816] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:27:31 2022] DELETE /identity/v3/users/90b16e382eb04746ba775a9372dd8149 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 19:27:32.576193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b377b-590b-48b4-aa8e-676c2d4d52b4 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:32.578048 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-110b377b-590b-48b4-aa8e-676c2d4d52b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:32.591307 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b377b-590b-48b4-aa8e-676c2d4d52b4 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:32.648018 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa9b4b-fc65-4cfe-814e-41412b65e372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:32.649404 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa9b4b-fc65-4cfe-814e-41412b65e372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:32.649691 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa9b4b-fc65-4cfe-814e-41412b65e372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:32.649966 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa9b4b-fc65-4cfe-814e-41412b65e372 admin admin] PATH_INFO: `/v3/projects/d9f16982615f4dbf8ed46a3ec038f65e` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:32.651144 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa9b4b-fc65-4cfe-814e-41412b65e372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f16982615f4dbf8ed46a3ec038f65e)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:32.653076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa9b4b-fc65-4cfe-814e-41412b65e372 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:32.751197 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cbfa9b4b-fc65-4cfe-814e-41412b65e372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f16982615f4dbf8ed46a3ec038f65e', 'request_id': 'req-cbfa9b4b-fc65-4cfe-814e-41412b65e372'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:32.806309 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cbfa9b4b-fc65-4cfe-814e-41412b65e372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f16982615f4dbf8ed46a3ec038f65e', 'request_id': 'req-cbfa9b4b-fc65-4cfe-814e-41412b65e372'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:32.812253 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cbfa9b4b-fc65-4cfe-814e-41412b65e372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f16982615f4dbf8ed46a3ec038f65e', 'request_id': 'req-cbfa9b4b-fc65-4cfe-814e-41412b65e372'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:32.823969 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cbfa9b4b-fc65-4cfe-814e-41412b65e372 admin admin] The token cache is being invalidate because project d9f16982615f4dbf8ed46a3ec038f65e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:27:32.824257 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cbfa9b4b-fc65-4cfe-814e-41412b65e372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:32.824942 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cbfa9b4b-fc65-4cfe-814e-41412b65e372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:32.826495 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1426/2817] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:27:32 2022] DELETE /identity/v3/projects/d9f16982615f4dbf8ed46a3ec038f65e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:32.831755 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c348b4c-110e-4586-ba76-68a471bb6cf6 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:32.832098 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c348b4c-110e-4586-ba76-68a471bb6cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:32.832386 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c348b4c-110e-4586-ba76-68a471bb6cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:32.851640 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-8c348b4c-110e-4586-ba76-68a471bb6cf6 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:32.852849 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8c348b4c-110e-4586-ba76-68a471bb6cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:32.861654 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:32.912109 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1392/2818] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:27:32.919860 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb0607-2d6b-423f-8bda-3b3704d908e9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:32.920173 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb0607-2d6b-423f-8bda-3b3704d908e9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:32.920452 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb0607-2d6b-423f-8bda-3b3704d908e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:32.938638 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-bcdb0607-2d6b-423f-8bda-3b3704d908e9 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:32.939923 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bcdb0607-2d6b-423f-8bda-3b3704d908e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:32.948359 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1427/2819] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:27:32.952637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c348b4c-110e-4586-ba76-68a471bb6cf6 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:32.962733 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c348b4c-110e-4586-ba76-68a471bb6cf6 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:32.967981 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9cad0-829c-400f-92dc-8a55ca76214b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:32.968644 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9cad0-829c-400f-92dc-8a55ca76214b admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:32.968933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9cad0-829c-400f-92dc-8a55ca76214b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:32.969215 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9cad0-829c-400f-92dc-8a55ca76214b admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:32.969977 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9cad0-829c-400f-92dc-8a55ca76214b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:27:32.970339 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9cad0-829c-400f-92dc-8a55ca76214b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:32.971987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9cad0-829c-400f-92dc-8a55ca76214b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:32.983595 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1393/2820] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:27:32.987841 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb0607-2d6b-423f-8bda-3b3704d908e9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:32.997064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb0607-2d6b-423f-8bda-3b3704d908e9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:33.001832 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd6864-8705-4838-96be-ceeb0565263d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:33.002436 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd6864-8705-4838-96be-ceeb0565263d admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:33.002726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd6864-8705-4838-96be-ceeb0565263d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:33.003005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd6864-8705-4838-96be-ceeb0565263d admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:33.003771 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd6864-8705-4838-96be-ceeb0565263d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:33.005784 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd6864-8705-4838-96be-ceeb0565263d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:33.009756 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:33.107330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1428/2821] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:27:32 2022] POST /identity/v3/projects => generated 407 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:27:33.111582 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9cad0-829c-400f-92dc-8a55ca76214b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:33.122213 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9cad0-829c-400f-92dc-8a55ca76214b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:33.178933 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f7566-b86b-4fc5-a652-7b861577d528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:33.179625 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f7566-b86b-4fc5-a652-7b861577d528 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:33.179912 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f7566-b86b-4fc5-a652-7b861577d528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:33.180220 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f7566-b86b-4fc5-a652-7b861577d528 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:33.180912 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f7566-b86b-4fc5-a652-7b861577d528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:33.182833 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f7566-b86b-4fc5-a652-7b861577d528 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:33.306133 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1394/2822] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:27:33 2022] POST /identity/v3/users => generated 304 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:27:33.310181 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd6864-8705-4838-96be-ceeb0565263d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:33.320301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd6864-8705-4838-96be-ceeb0565263d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:33.324783 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdf684-97c2-4348-9f1d-382e34cacbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:33.325421 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdf684-97c2-4348-9f1d-382e34cacbfd admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:33.325706 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdf684-97c2-4348-9f1d-382e34cacbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:33.325984 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdf684-97c2-4348-9f1d-382e34cacbfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:33.326711 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdf684-97c2-4348-9f1d-382e34cacbfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:33.328391 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdf684-97c2-4348-9f1d-382e34cacbfd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:33.339256 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1429/2823] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:27:33.343217 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f7566-b86b-4fc5-a652-7b861577d528 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:33.353760 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f7566-b86b-4fc5-a652-7b861577d528 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:33.358281 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdc7d3-c45e-4b97-ae1f-227d9c33f73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:33.358932 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdc7d3-c45e-4b97-ae1f-227d9c33f73c admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:33.359251 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdc7d3-c45e-4b97-ae1f-227d9c33f73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:33.359535 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdc7d3-c45e-4b97-ae1f-227d9c33f73c admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:33.360250 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdc7d3-c45e-4b97-ae1f-227d9c33f73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:33.361837 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdc7d3-c45e-4b97-ae1f-227d9c33f73c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:33.371643 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1395/2824] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:27:33.375684 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdf684-97c2-4348-9f1d-382e34cacbfd admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:33.385477 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdf684-97c2-4348-9f1d-382e34cacbfd admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:33.390041 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834eaba-ddb7-44e2-a2fa-7a11838d053a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:33.390912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834eaba-ddb7-44e2-a2fa-7a11838d053a admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:33.391227 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834eaba-ddb7-44e2-a2fa-7a11838d053a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:33.391515 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834eaba-ddb7-44e2-a2fa-7a11838d053a admin admin] PATH_INFO: `/v3/projects/9896f0422d2b47b9b70f9911da7f3816/users/ba5e6cd2b05740ec861139581fb2af4e/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:33.406608 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834eaba-ddb7-44e2-a2fa-7a11838d053a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9896f0422d2b47b9b70f9911da7f3816, user_id=ba5e6cd2b05740ec861139581fb2af4e, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:33.409289 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834eaba-ddb7-44e2-a2fa-7a11838d053a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:33.505763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1430/2825] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:27:33 2022] PUT /identity/v3/projects/9896f0422d2b47b9b70f9911da7f3816/users/ba5e6cd2b05740ec861139581fb2af4e/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:33.517455 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ddc81-5fd3-4683-97e6-0061d18d387b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:33.517749 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ddc81-5fd3-4683-97e6-0061d18d387b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:33.518031 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ddc81-5fd3-4683-97e6-0061d18d387b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:33.544308 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-582ddc81-5fd3-4683-97e6-0061d18d387b None None] MFA Rules not processed for user `ba5e6cd2b05740ec861139581fb2af4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:33.545496 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-582ddc81-5fd3-4683-97e6-0061d18d387b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:33.607509 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1396/2826] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:33 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:27:33.611998 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834eaba-ddb7-44e2-a2fa-7a11838d053a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:33.676629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834eaba-ddb7-44e2-a2fa-7a11838d053a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:33.681396 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390cae7-fd26-435f-aac5-5ce909de4f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:33.682058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390cae7-fd26-435f-aac5-5ce909de4f41 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:33.682343 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390cae7-fd26-435f-aac5-5ce909de4f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:33.682620 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390cae7-fd26-435f-aac5-5ce909de4f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:33.683361 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390cae7-fd26-435f-aac5-5ce909de4f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:33.685516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390cae7-fd26-435f-aac5-5ce909de4f41 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:33.809759 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1431/2827] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:27:33 2022] POST /identity/v3/projects => generated 407 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:27:33.813798 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ddc81-5fd3-4683-97e6-0061d18d387b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:33.824921 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ddc81-5fd3-4683-97e6-0061d18d387b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:33.829499 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:33.880770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c31b1d-f06d-43f6-a785-21d55ba6dafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:33.881469 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c31b1d-f06d-43f6-a785-21d55ba6dafa admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:33.881759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c31b1d-f06d-43f6-a785-21d55ba6dafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:33.882042 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c31b1d-f06d-43f6-a785-21d55ba6dafa admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:33.882745 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c31b1d-f06d-43f6-a785-21d55ba6dafa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:33.885321 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c31b1d-f06d-43f6-a785-21d55ba6dafa admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:33.957120 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1397/2828] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:27:33 2022] POST /identity/v3/users => generated 310 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:27:33.961095 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390cae7-fd26-435f-aac5-5ce909de4f41 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:33.971611 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390cae7-fd26-435f-aac5-5ce909de4f41 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:33.976431 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8a816-1dc6-4afd-abed-2e6e1d150481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:33.977119 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8a816-1dc6-4afd-abed-2e6e1d150481 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:33.977413 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8a816-1dc6-4afd-abed-2e6e1d150481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:33.977693 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8a816-1dc6-4afd-abed-2e6e1d150481 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:33.978421 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8a816-1dc6-4afd-abed-2e6e1d150481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:33.980085 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8a816-1dc6-4afd-abed-2e6e1d150481 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:33.991052 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1432/2829] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:27:33.995948 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c31b1d-f06d-43f6-a785-21d55ba6dafa admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:34.006229 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c31b1d-f06d-43f6-a785-21d55ba6dafa admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:34.012097 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd48af1-003b-4949-9d3c-7c49c060107b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:34.012999 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd48af1-003b-4949-9d3c-7c49c060107b admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:34.013605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd48af1-003b-4949-9d3c-7c49c060107b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:34.013605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd48af1-003b-4949-9d3c-7c49c060107b admin admin] PATH_INFO: `/v3/projects/bb6749944c9c497796fa71a0618c96c5/users/7919e2fa702b432480c0097cfa0627e5/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:34.029934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd48af1-003b-4949-9d3c-7c49c060107b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb6749944c9c497796fa71a0618c96c5, user_id=7919e2fa702b432480c0097cfa0627e5, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:34.033044 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd48af1-003b-4949-9d3c-7c49c060107b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:34.111963 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1398/2830] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:27:33 2022] PUT /identity/v3/projects/bb6749944c9c497796fa71a0618c96c5/users/7919e2fa702b432480c0097cfa0627e5/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:34.116029 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8a816-1dc6-4afd-abed-2e6e1d150481 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:34.126623 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8a816-1dc6-4afd-abed-2e6e1d150481 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:34.182083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c7802-8169-4ecf-b738-86628005cb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:34.182678 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c7802-8169-4ecf-b738-86628005cb7f admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:34.182966 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c7802-8169-4ecf-b738-86628005cb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:34.183281 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c7802-8169-4ecf-b738-86628005cb7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:34.184000 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c7802-8169-4ecf-b738-86628005cb7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:34.185597 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c7802-8169-4ecf-b738-86628005cb7f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:34.195122 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1433/2831] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:27:34.199969 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd48af1-003b-4949-9d3c-7c49c060107b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:34.210275 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd48af1-003b-4949-9d3c-7c49c060107b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:34.216241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5069d6-6e16-4240-a599-cda1a4cc3f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:34.217127 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5069d6-6e16-4240-a599-cda1a4cc3f29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:34.217414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5069d6-6e16-4240-a599-cda1a4cc3f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:34.217689 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5069d6-6e16-4240-a599-cda1a4cc3f29 admin admin] PATH_INFO: `/v3/projects/bb6749944c9c497796fa71a0618c96c5/users/7919e2fa702b432480c0097cfa0627e5/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:34.222242 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5069d6-6e16-4240-a599-cda1a4cc3f29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb6749944c9c497796fa71a0618c96c5, user_id=7919e2fa702b432480c0097cfa0627e5, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:34.222484 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5069d6-6e16-4240-a599-cda1a4cc3f29 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:34.233436 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1399/2832] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:27:34 2022] PUT /identity/v3/projects/bb6749944c9c497796fa71a0618c96c5/users/7919e2fa702b432480c0097cfa0627e5/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:34.244317 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d192c-f538-4093-a6c1-7f1e1493d2cf None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:34.248342 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d192c-f538-4093-a6c1-7f1e1493d2cf None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:34.248342 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d192c-f538-4093-a6c1-7f1e1493d2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:34.270560 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-735d192c-f538-4093-a6c1-7f1e1493d2cf None None] MFA Rules not processed for user `7919e2fa702b432480c0097cfa0627e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:34.271800 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-735d192c-f538-4093-a6c1-7f1e1493d2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:34.279763 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:34.340950 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1434/2833] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:34 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:27:34.351577 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5069d6-6e16-4240-a599-cda1a4cc3f29 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:34.353990 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-bb5069d6-6e16-4240-a599-cda1a4cc3f29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:34.410633 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5069d6-6e16-4240-a599-cda1a4cc3f29 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:34.414236 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb82f11-b33b-4135-a38a-4a58b2cf7187 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:34.415469 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb82f11-b33b-4135-a38a-4a58b2cf7187 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:34.415752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb82f11-b33b-4135-a38a-4a58b2cf7187 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:34.416027 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb82f11-b33b-4135-a38a-4a58b2cf7187 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:34.424911 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb82f11-b33b-4135-a38a-4a58b2cf7187 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:34.426335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb82f11-b33b-4135-a38a-4a58b2cf7187 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:34.436741 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1400/2834] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:27:34 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:27:34.783474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d192c-f538-4093-a6c1-7f1e1493d2cf None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:34.793844 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d192c-f538-4093-a6c1-7f1e1493d2cf None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:34.797454 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3016e8-9033-45f8-9a62-d7b5f1a4720e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:34.798092 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3016e8-9033-45f8-9a62-d7b5f1a4720e service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:34.798379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3016e8-9033-45f8-9a62-d7b5f1a4720e service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:34.798660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3016e8-9033-45f8-9a62-d7b5f1a4720e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:34.854384 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3016e8-9033-45f8-9a62-d7b5f1a4720e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:34.855837 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3016e8-9033-45f8-9a62-d7b5f1a4720e service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:34.864557 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1435/2835] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:27:34 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:27:44.432363 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=66,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:44.860567 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=53 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:45.689647 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb82f11-b33b-4135-a38a-4a58b2cf7187 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:45.750299 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb82f11-b33b-4135-a38a-4a58b2cf7187 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:45.754854 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a8e33-86df-4757-a9a6-98d6cc6c8fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:45.755770 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a8e33-86df-4757-a9a6-98d6cc6c8fe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:45.756059 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a8e33-86df-4757-a9a6-98d6cc6c8fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:45.756340 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a8e33-86df-4757-a9a6-98d6cc6c8fe5 admin admin] PATH_INFO: `/v3/users/ba5e6cd2b05740ec861139581fb2af4e` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:45.757557 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a8e33-86df-4757-a9a6-98d6cc6c8fe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba5e6cd2b05740ec861139581fb2af4e)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:45.760029 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a8e33-86df-4757-a9a6-98d6cc6c8fe5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:45.933700 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3e2a8e33-86df-4757-a9a6-98d6cc6c8fe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba5e6cd2b05740ec861139581fb2af4e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:45.940588 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3e2a8e33-86df-4757-a9a6-98d6cc6c8fe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba5e6cd2b05740ec861139581fb2af4e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:45.941257 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3e2a8e33-86df-4757-a9a6-98d6cc6c8fe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba5e6cd2b05740ec861139581fb2af4e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:45.941877 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3e2a8e33-86df-4757-a9a6-98d6cc6c8fe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba5e6cd2b05740ec861139581fb2af4e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:45.953580 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3e2a8e33-86df-4757-a9a6-98d6cc6c8fe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba5e6cd2b05740ec861139581fb2af4e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:46.011801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1401/2836] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:27:45 2022] DELETE /identity/v3/users/ba5e6cd2b05740ec861139581fb2af4e => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:46.016436 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a69952-2766-444d-8702-ddd84b9364c6 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:46.016884 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a69952-2766-444d-8702-ddd84b9364c6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:46.017076 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a69952-2766-444d-8702-ddd84b9364c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:46.034653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-29a69952-2766-444d-8702-ddd84b9364c6 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:46.035881 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-29a69952-2766-444d-8702-ddd84b9364c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:46.093331 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1436/2837] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:27:46.103739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a8e33-86df-4757-a9a6-98d6cc6c8fe5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:46.105130 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3e2a8e33-86df-4757-a9a6-98d6cc6c8fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:46.163702 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a8e33-86df-4757-a9a6-98d6cc6c8fe5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:46.167978 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1dc2b5-7eb7-42fe-a61d-0ce888ca634d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:46.168644 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1dc2b5-7eb7-42fe-a61d-0ce888ca634d service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:46.168930 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1dc2b5-7eb7-42fe-a61d-0ce888ca634d service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:46.169207 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1dc2b5-7eb7-42fe-a61d-0ce888ca634d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:46.178111 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1dc2b5-7eb7-42fe-a61d-0ce888ca634d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:46.179602 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1dc2b5-7eb7-42fe-a61d-0ce888ca634d service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:46.188992 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1402/2838] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:27:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:27:46.584505 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a69952-2766-444d-8702-ddd84b9364c6 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:46.586659 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-29a69952-2766-444d-8702-ddd84b9364c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:46.602019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a69952-2766-444d-8702-ddd84b9364c6 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:46.607242 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ff3ed-ba7b-4246-a854-d11e4e102dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:46.611118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ff3ed-ba7b-4246-a854-d11e4e102dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:46.611118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ff3ed-ba7b-4246-a854-d11e4e102dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:46.611118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ff3ed-ba7b-4246-a854-d11e4e102dad admin admin] PATH_INFO: `/v3/projects/9896f0422d2b47b9b70f9911da7f3816` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:46.611118 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ff3ed-ba7b-4246-a854-d11e4e102dad admin admin] RBAC: Authorizing `identity:delete_project(project_id=9896f0422d2b47b9b70f9911da7f3816)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:46.612206 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ff3ed-ba7b-4246-a854-d11e4e102dad admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:46.731247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ae6ff3ed-ba7b-4246-a854-d11e4e102dad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9896f0422d2b47b9b70f9911da7f3816', 'request_id': 'req-ae6ff3ed-ba7b-4246-a854-d11e4e102dad'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:46.785163 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ae6ff3ed-ba7b-4246-a854-d11e4e102dad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9896f0422d2b47b9b70f9911da7f3816', 'request_id': 'req-ae6ff3ed-ba7b-4246-a854-d11e4e102dad'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:46.790817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ae6ff3ed-ba7b-4246-a854-d11e4e102dad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9896f0422d2b47b9b70f9911da7f3816', 'request_id': 'req-ae6ff3ed-ba7b-4246-a854-d11e4e102dad'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:46.801677 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ae6ff3ed-ba7b-4246-a854-d11e4e102dad admin admin] The token cache is being invalidate because project 9896f0422d2b47b9b70f9911da7f3816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:27:46.806174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ae6ff3ed-ba7b-4246-a854-d11e4e102dad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:46.806174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ae6ff3ed-ba7b-4246-a854-d11e4e102dad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:46.806174 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1437/2839] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:27:46 2022] DELETE /identity/v3/projects/9896f0422d2b47b9b70f9911da7f3816 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:46.808582 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1dc2b5-7eb7-42fe-a61d-0ce888ca634d service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:46.818106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1dc2b5-7eb7-42fe-a61d-0ce888ca634d service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:46.836308 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:46.875279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe208ab-4820-4cec-9eac-9c869086c224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:46.876150 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe208ab-4820-4cec-9eac-9c869086c224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:46.876467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe208ab-4820-4cec-9eac-9c869086c224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:46.876750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe208ab-4820-4cec-9eac-9c869086c224 admin admin] PATH_INFO: `/v3/users/7919e2fa702b432480c0097cfa0627e5` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:46.878014 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe208ab-4820-4cec-9eac-9c869086c224 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7919e2fa702b432480c0097cfa0627e5)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:46.880168 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe208ab-4820-4cec-9eac-9c869086c224 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:47.098972 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-fbe208ab-4820-4cec-9eac-9c869086c224 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7919e2fa702b432480c0097cfa0627e5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:47.106320 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-fbe208ab-4820-4cec-9eac-9c869086c224 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7919e2fa702b432480c0097cfa0627e5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:47.106987 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-fbe208ab-4820-4cec-9eac-9c869086c224 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7919e2fa702b432480c0097cfa0627e5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:47.107635 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-fbe208ab-4820-4cec-9eac-9c869086c224 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7919e2fa702b432480c0097cfa0627e5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:47.119965 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-fbe208ab-4820-4cec-9eac-9c869086c224 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7919e2fa702b432480c0097cfa0627e5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:47.183383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1403/2840] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:27:46 2022] DELETE /identity/v3/users/7919e2fa702b432480c0097cfa0627e5 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:47.587550 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ff3ed-ba7b-4246-a854-d11e4e102dad admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:47.589022 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ae6ff3ed-ba7b-4246-a854-d11e4e102dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:47.605555 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ff3ed-ba7b-4246-a854-d11e4e102dad admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:47.661770 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7b5dc-2fcd-4f32-9cc3-1a082474bf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:47.662657 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7b5dc-2fcd-4f32-9cc3-1a082474bf25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:47.662941 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7b5dc-2fcd-4f32-9cc3-1a082474bf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:47.663247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7b5dc-2fcd-4f32-9cc3-1a082474bf25 admin admin] PATH_INFO: `/v3/projects/bb6749944c9c497796fa71a0618c96c5` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:47.664429 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7b5dc-2fcd-4f32-9cc3-1a082474bf25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb6749944c9c497796fa71a0618c96c5)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:47.667090 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7b5dc-2fcd-4f32-9cc3-1a082474bf25 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:47.761764 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:47.777386 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-46a7b5dc-2fcd-4f32-9cc3-1a082474bf25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb6749944c9c497796fa71a0618c96c5', 'request_id': 'req-46a7b5dc-2fcd-4f32-9cc3-1a082474bf25'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:47.891727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-46a7b5dc-2fcd-4f32-9cc3-1a082474bf25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb6749944c9c497796fa71a0618c96c5', 'request_id': 'req-46a7b5dc-2fcd-4f32-9cc3-1a082474bf25'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:47.897539 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-46a7b5dc-2fcd-4f32-9cc3-1a082474bf25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb6749944c9c497796fa71a0618c96c5', 'request_id': 'req-46a7b5dc-2fcd-4f32-9cc3-1a082474bf25'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:47.909316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-46a7b5dc-2fcd-4f32-9cc3-1a082474bf25 admin admin] The token cache is being invalidate because project bb6749944c9c497796fa71a0618c96c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:27:47.909674 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-46a7b5dc-2fcd-4f32-9cc3-1a082474bf25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:47.910345 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-46a7b5dc-2fcd-4f32-9cc3-1a082474bf25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:47.911917 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1438/2841] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:27:47 2022] DELETE /identity/v3/projects/bb6749944c9c497796fa71a0618c96c5 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:47.916783 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ebdea-3289-4cb1-94a2-e4dcdb07ea1d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:47.917252 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ebdea-3289-4cb1-94a2-e4dcdb07ea1d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:47.917354 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ebdea-3289-4cb1-94a2-e4dcdb07ea1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:47.935237 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-563ebdea-3289-4cb1-94a2-e4dcdb07ea1d None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:47.936445 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-563ebdea-3289-4cb1-94a2-e4dcdb07ea1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:47.996510 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1404/2842] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:27:48.002858 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9eda68-10f0-49b1-9737-a4a25c72046c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:48.003236 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9eda68-10f0-49b1-9737-a4a25c72046c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:48.003509 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9eda68-10f0-49b1-9737-a4a25c72046c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:48.022005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-db9eda68-10f0-49b1-9737-a4a25c72046c None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:48.023221 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-db9eda68-10f0-49b1-9737-a4a25c72046c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:48.031988 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1439/2843] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:27:48.036160 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ebdea-3289-4cb1-94a2-e4dcdb07ea1d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:48.045714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ebdea-3289-4cb1-94a2-e4dcdb07ea1d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:48.050262 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216ae87-4877-46ef-a306-7a4eeb5ebb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:48.050913 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216ae87-4877-46ef-a306-7a4eeb5ebb55 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:48.051225 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216ae87-4877-46ef-a306-7a4eeb5ebb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:48.051508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216ae87-4877-46ef-a306-7a4eeb5ebb55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:48.052240 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d216ae87-4877-46ef-a306-7a4eeb5ebb55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:27:48.052556 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d216ae87-4877-46ef-a306-7a4eeb5ebb55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:48.054179 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d216ae87-4877-46ef-a306-7a4eeb5ebb55 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:48.066880 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1405/2844] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:27:48.074216 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9eda68-10f0-49b1-9737-a4a25c72046c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:48.083622 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9eda68-10f0-49b1-9737-a4a25c72046c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:48.088270 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ce5f0-df2d-44b2-9315-187cc84e8b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:48.088912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ce5f0-df2d-44b2-9315-187cc84e8b5d admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:48.089196 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ce5f0-df2d-44b2-9315-187cc84e8b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:48.089469 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ce5f0-df2d-44b2-9315-187cc84e8b5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:48.090146 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ce5f0-df2d-44b2-9315-187cc84e8b5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:48.092185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ce5f0-df2d-44b2-9315-187cc84e8b5d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:48.156517 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1440/2845] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:27:48 2022] POST /identity/v3/projects => generated 387 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:27:48.160672 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216ae87-4877-46ef-a306-7a4eeb5ebb55 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:48.170685 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216ae87-4877-46ef-a306-7a4eeb5ebb55 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:48.197062 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:48.226667 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d847a-6517-4d22-9b7b-9d922cfb1c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:48.227413 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d847a-6517-4d22-9b7b-9d922cfb1c1d admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:48.227695 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d847a-6517-4d22-9b7b-9d922cfb1c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:48.231184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d847a-6517-4d22-9b7b-9d922cfb1c1d admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:48.231184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d847a-6517-4d22-9b7b-9d922cfb1c1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:48.231184 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d847a-6517-4d22-9b7b-9d922cfb1c1d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:48.326296 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1406/2846] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:27:48 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:27:48.330294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ce5f0-df2d-44b2-9315-187cc84e8b5d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:48.340486 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ce5f0-df2d-44b2-9315-187cc84e8b5d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:48.345249 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f040a0f-3231-4300-a46e-fd76322bd8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:48.345974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f040a0f-3231-4300-a46e-fd76322bd8f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:48.346266 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f040a0f-3231-4300-a46e-fd76322bd8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:48.346553 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f040a0f-3231-4300-a46e-fd76322bd8f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:48.347314 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f040a0f-3231-4300-a46e-fd76322bd8f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:48.348948 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f040a0f-3231-4300-a46e-fd76322bd8f8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:48.359510 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1441/2847] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:27:48.363325 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d847a-6517-4d22-9b7b-9d922cfb1c1d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:48.373541 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d847a-6517-4d22-9b7b-9d922cfb1c1d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:48.378075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b9846-c51f-4067-98d5-38a573fa4f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:48.378718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b9846-c51f-4067-98d5-38a573fa4f63 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:48.379003 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b9846-c51f-4067-98d5-38a573fa4f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:48.379313 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b9846-c51f-4067-98d5-38a573fa4f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:48.380031 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b9846-c51f-4067-98d5-38a573fa4f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:48.381611 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b9846-c51f-4067-98d5-38a573fa4f63 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:48.392172 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1407/2848] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 19:27:48.396026 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f040a0f-3231-4300-a46e-fd76322bd8f8 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:48.406609 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f040a0f-3231-4300-a46e-fd76322bd8f8 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:48.411058 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fd48e-209f-48d3-8fa2-dba48b67fa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:48.411956 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fd48e-209f-48d3-8fa2-dba48b67fa9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:48.412238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fd48e-209f-48d3-8fa2-dba48b67fa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:48.412509 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fd48e-209f-48d3-8fa2-dba48b67fa9e admin admin] PATH_INFO: `/v3/projects/74096580aacb413b8b8f1e33f466de51/users/3429e8069375473ba1f646b6142009d8/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:48.427092 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953fd48e-209f-48d3-8fa2-dba48b67fa9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=74096580aacb413b8b8f1e33f466de51, user_id=3429e8069375473ba1f646b6142009d8, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:48.429741 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953fd48e-209f-48d3-8fa2-dba48b67fa9e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:48.522462 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1442/2849] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:27:48 2022] PUT /identity/v3/projects/74096580aacb413b8b8f1e33f466de51/users/3429e8069375473ba1f646b6142009d8/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:48.583259 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2df4ba-69b4-4335-bcd6-160ea43cbcb5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:48.583555 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2df4ba-69b4-4335-bcd6-160ea43cbcb5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:48.583837 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2df4ba-69b4-4335-bcd6-160ea43cbcb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:48.609490 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-5e2df4ba-69b4-4335-bcd6-160ea43cbcb5 None None] MFA Rules not processed for user `3429e8069375473ba1f646b6142009d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:48.610691 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5e2df4ba-69b4-4335-bcd6-160ea43cbcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:48.673335 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1408/2850] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:27:48.677639 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fd48e-209f-48d3-8fa2-dba48b67fa9e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:48.737996 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fd48e-209f-48d3-8fa2-dba48b67fa9e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:48.742629 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d1c27-017b-4928-81bb-2db20e5ff1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:48.743968 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d1c27-017b-4928-81bb-2db20e5ff1f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:48.744255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d1c27-017b-4928-81bb-2db20e5ff1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:48.744535 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d1c27-017b-4928-81bb-2db20e5ff1f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:48.745231 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d1c27-017b-4928-81bb-2db20e5ff1f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:48.747141 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d1c27-017b-4928-81bb-2db20e5ff1f2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:48.896002 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1443/2851] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:27:48 2022] POST /identity/v3/projects => generated 387 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:27:48.900486 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2df4ba-69b4-4335-bcd6-160ea43cbcb5 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:48.911309 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2df4ba-69b4-4335-bcd6-160ea43cbcb5 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:48.966279 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ff61d-9f8d-4743-b375-aec84ca92cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:48.966903 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ff61d-9f8d-4743-b375-aec84ca92cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:48.967146 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ff61d-9f8d-4743-b375-aec84ca92cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:48.967455 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ff61d-9f8d-4743-b375-aec84ca92cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:48.968142 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ff61d-9f8d-4743-b375-aec84ca92cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:48.970058 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ff61d-9f8d-4743-b375-aec84ca92cb7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:49.041934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1409/2852] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:27:48 2022] POST /identity/v3/users => generated 300 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:27:49.045975 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d1c27-017b-4928-81bb-2db20e5ff1f2 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:49.059642 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d1c27-017b-4928-81bb-2db20e5ff1f2 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:49.064273 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8779fc4-aa0a-480e-8bf3-f100e769477b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:49.064929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8779fc4-aa0a-480e-8bf3-f100e769477b admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:49.065218 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8779fc4-aa0a-480e-8bf3-f100e769477b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:49.065528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8779fc4-aa0a-480e-8bf3-f100e769477b admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:49.066243 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8779fc4-aa0a-480e-8bf3-f100e769477b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:49.067890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8779fc4-aa0a-480e-8bf3-f100e769477b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:49.078339 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1444/2853] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:27:49.082380 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ff61d-9f8d-4743-b375-aec84ca92cb7 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:49.093002 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ff61d-9f8d-4743-b375-aec84ca92cb7 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:49.097559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fce0a-52af-4853-bba3-a466bd27c412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:49.098428 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fce0a-52af-4853-bba3-a466bd27c412 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:49.098712 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fce0a-52af-4853-bba3-a466bd27c412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:49.098989 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fce0a-52af-4853-bba3-a466bd27c412 admin admin] PATH_INFO: `/v3/projects/0149f518784d4f6285de948b08383cca/users/0acd01f442c44801a76c1fe952743e65/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:49.113885 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fce0a-52af-4853-bba3-a466bd27c412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0149f518784d4f6285de948b08383cca, user_id=0acd01f442c44801a76c1fe952743e65, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:49.116581 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fce0a-52af-4853-bba3-a466bd27c412 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:49.228744 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1410/2854] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:27:49 2022] PUT /identity/v3/projects/0149f518784d4f6285de948b08383cca/users/0acd01f442c44801a76c1fe952743e65/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:49.232513 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8779fc4-aa0a-480e-8bf3-f100e769477b admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:49.242599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8779fc4-aa0a-480e-8bf3-f100e769477b admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:49.262443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:49.298703 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d83317-53d6-4611-95b3-1f4ffcb21c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:49.299389 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d83317-53d6-4611-95b3-1f4ffcb21c4f admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:49.299676 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d83317-53d6-4611-95b3-1f4ffcb21c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:49.299951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d83317-53d6-4611-95b3-1f4ffcb21c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:49.300675 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d83317-53d6-4611-95b3-1f4ffcb21c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:49.302289 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d83317-53d6-4611-95b3-1f4ffcb21c4f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:49.311817 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1445/2855] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:27:49.315894 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fce0a-52af-4853-bba3-a466bd27c412 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:49.321819 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:49.327048 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fce0a-52af-4853-bba3-a466bd27c412 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:49.331653 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fa7e9-231f-486e-9751-2bf6617df901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:49.332536 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fa7e9-231f-486e-9751-2bf6617df901 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:49.332822 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fa7e9-231f-486e-9751-2bf6617df901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:49.333143 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fa7e9-231f-486e-9751-2bf6617df901 admin admin] PATH_INFO: `/v3/projects/0149f518784d4f6285de948b08383cca/users/0acd01f442c44801a76c1fe952743e65/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:49.335189 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fa7e9-231f-486e-9751-2bf6617df901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0149f518784d4f6285de948b08383cca, user_id=0acd01f442c44801a76c1fe952743e65, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:49.338563 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fa7e9-231f-486e-9751-2bf6617df901 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:49.349383 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1411/2856] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:27:49 2022] PUT /identity/v3/projects/0149f518784d4f6285de948b08383cca/users/0acd01f442c44801a76c1fe952743e65/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:49.359593 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388797e-d70e-40c0-b02e-a89312c83dd0 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:49.359897 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388797e-d70e-40c0-b02e-a89312c83dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:49.360185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388797e-d70e-40c0-b02e-a89312c83dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:49.386600 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5388797e-d70e-40c0-b02e-a89312c83dd0 None None] MFA Rules not processed for user `0acd01f442c44801a76c1fe952743e65`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:49.387870 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5388797e-d70e-40c0-b02e-a89312c83dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:49.455558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1446/2857] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:27:49.466860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fa7e9-231f-486e-9751-2bf6617df901 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:49.468530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-836fa7e9-231f-486e-9751-2bf6617df901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:49.525473 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fa7e9-231f-486e-9751-2bf6617df901 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:49.528973 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7de616-af44-4d4f-8ae3-3f0a532e47c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:49.529605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7de616-af44-4d4f-8ae3-3f0a532e47c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:49.529890 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7de616-af44-4d4f-8ae3-3f0a532e47c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:49.530432 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7de616-af44-4d4f-8ae3-3f0a532e47c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:49.539668 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7de616-af44-4d4f-8ae3-3f0a532e47c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:49.541168 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7de616-af44-4d4f-8ae3-3f0a532e47c1 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:49.550471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1412/2858] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:27:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:27:50.655005 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388797e-d70e-40c0-b02e-a89312c83dd0 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:50.716011 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388797e-d70e-40c0-b02e-a89312c83dd0 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:50.720463 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280c055-9544-44b6-b008-edc8517986c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:50.721319 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280c055-9544-44b6-b008-edc8517986c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:50.721604 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280c055-9544-44b6-b008-edc8517986c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:50.722023 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280c055-9544-44b6-b008-edc8517986c2 admin admin] PATH_INFO: `/v3/users/3429e8069375473ba1f646b6142009d8` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:50.723220 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4280c055-9544-44b6-b008-edc8517986c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3429e8069375473ba1f646b6142009d8)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:50.725132 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4280c055-9544-44b6-b008-edc8517986c2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:50.864936 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4280c055-9544-44b6-b008-edc8517986c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3429e8069375473ba1f646b6142009d8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:50.865590 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4280c055-9544-44b6-b008-edc8517986c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3429e8069375473ba1f646b6142009d8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:50.866723 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:50.877874 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4280c055-9544-44b6-b008-edc8517986c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3429e8069375473ba1f646b6142009d8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:50.878530 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4280c055-9544-44b6-b008-edc8517986c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3429e8069375473ba1f646b6142009d8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:50.886171 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4280c055-9544-44b6-b008-edc8517986c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3429e8069375473ba1f646b6142009d8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:50.956460 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1447/2859] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:27:50 2022] DELETE /identity/v3/users/3429e8069375473ba1f646b6142009d8 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:50.960991 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0474d-dd50-49fb-a545-4ff831c4a42e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:50.961488 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0474d-dd50-49fb-a545-4ff831c4a42e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:50.962263 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0474d-dd50-49fb-a545-4ff831c4a42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:50.978912 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d0f0474d-dd50-49fb-a545-4ff831c4a42e None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:50.980158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d0f0474d-dd50-49fb-a545-4ff831c4a42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:51.038530 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1413/2860] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:27:51.048262 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280c055-9544-44b6-b008-edc8517986c2 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:51.049694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4280c055-9544-44b6-b008-edc8517986c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:51.106059 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280c055-9544-44b6-b008-edc8517986c2 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:51.109990 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99532d-c377-4104-9899-7f9cb778a3d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:51.110632 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99532d-c377-4104-9899-7f9cb778a3d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:51.110916 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99532d-c377-4104-9899-7f9cb778a3d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:51.111214 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99532d-c377-4104-9899-7f9cb778a3d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:51.120744 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99532d-c377-4104-9899-7f9cb778a3d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:51.122185 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99532d-c377-4104-9899-7f9cb778a3d2 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:51.131886 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1448/2861] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:27:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:27:51.533186 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0474d-dd50-49fb-a545-4ff831c4a42e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:51.535444 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d0f0474d-dd50-49fb-a545-4ff831c4a42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:51.550642 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0474d-dd50-49fb-a545-4ff831c4a42e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:51.557714 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382eeae-7c52-46d6-98d0-3c05bd949b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:51.559213 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382eeae-7c52-46d6-98d0-3c05bd949b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:51.559213 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382eeae-7c52-46d6-98d0-3c05bd949b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:51.559213 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382eeae-7c52-46d6-98d0-3c05bd949b27 admin admin] PATH_INFO: `/v3/projects/74096580aacb413b8b8f1e33f466de51` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:51.560433 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382eeae-7c52-46d6-98d0-3c05bd949b27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74096580aacb413b8b8f1e33f466de51)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:51.562360 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382eeae-7c52-46d6-98d0-3c05bd949b27 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:51.669559 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d382eeae-7c52-46d6-98d0-3c05bd949b27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74096580aacb413b8b8f1e33f466de51', 'request_id': 'req-d382eeae-7c52-46d6-98d0-3c05bd949b27'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:51.745523 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d382eeae-7c52-46d6-98d0-3c05bd949b27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74096580aacb413b8b8f1e33f466de51', 'request_id': 'req-d382eeae-7c52-46d6-98d0-3c05bd949b27'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:51.751241 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d382eeae-7c52-46d6-98d0-3c05bd949b27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74096580aacb413b8b8f1e33f466de51', 'request_id': 'req-d382eeae-7c52-46d6-98d0-3c05bd949b27'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:51.762230 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d382eeae-7c52-46d6-98d0-3c05bd949b27 admin admin] The token cache is being invalidate because project 74096580aacb413b8b8f1e33f466de51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:27:51.766411 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d382eeae-7c52-46d6-98d0-3c05bd949b27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:51.766411 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d382eeae-7c52-46d6-98d0-3c05bd949b27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:51.766411 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1414/2862] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:27:51 2022] DELETE /identity/v3/projects/74096580aacb413b8b8f1e33f466de51 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:51.768653 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99532d-c377-4104-9899-7f9cb778a3d2 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:51.778288 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99532d-c377-4104-9899-7f9cb778a3d2 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:51.833814 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c98442-a555-4ba0-9fc1-3665bf6702b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:51.834686 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c98442-a555-4ba0-9fc1-3665bf6702b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:51.834973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c98442-a555-4ba0-9fc1-3665bf6702b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:51.835287 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c98442-a555-4ba0-9fc1-3665bf6702b2 admin admin] PATH_INFO: `/v3/users/0acd01f442c44801a76c1fe952743e65` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:51.836961 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c98442-a555-4ba0-9fc1-3665bf6702b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0acd01f442c44801a76c1fe952743e65)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:51.838887 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c98442-a555-4ba0-9fc1-3665bf6702b2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:51.999705 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-97c98442-a555-4ba0-9fc1-3665bf6702b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0acd01f442c44801a76c1fe952743e65'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:52.000298 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-97c98442-a555-4ba0-9fc1-3665bf6702b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0acd01f442c44801a76c1fe952743e65'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:52.012385 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-97c98442-a555-4ba0-9fc1-3665bf6702b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0acd01f442c44801a76c1fe952743e65'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:52.018228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-97c98442-a555-4ba0-9fc1-3665bf6702b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0acd01f442c44801a76c1fe952743e65'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:52.018228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:52.020578 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-97c98442-a555-4ba0-9fc1-3665bf6702b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0acd01f442c44801a76c1fe952743e65'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:52.096820 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1449/2863] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:27:51 2022] DELETE /identity/v3/users/0acd01f442c44801a76c1fe952743e65 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:52.505081 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382eeae-7c52-46d6-98d0-3c05bd949b27 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:52.507420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d382eeae-7c52-46d6-98d0-3c05bd949b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:52.522467 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382eeae-7c52-46d6-98d0-3c05bd949b27 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:52.536992 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:52.582261 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea2507-bcc1-456f-bde5-3d6c6d7a1383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:52.587743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea2507-bcc1-456f-bde5-3d6c6d7a1383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:52.587743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea2507-bcc1-456f-bde5-3d6c6d7a1383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:52.587743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea2507-bcc1-456f-bde5-3d6c6d7a1383 admin admin] PATH_INFO: `/v3/projects/0149f518784d4f6285de948b08383cca` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:52.587743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea2507-bcc1-456f-bde5-3d6c6d7a1383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0149f518784d4f6285de948b08383cca)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:52.587743 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea2507-bcc1-456f-bde5-3d6c6d7a1383 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:52.676314 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-feea2507-bcc1-456f-bde5-3d6c6d7a1383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0149f518784d4f6285de948b08383cca', 'request_id': 'req-feea2507-bcc1-456f-bde5-3d6c6d7a1383'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:52.762557 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-feea2507-bcc1-456f-bde5-3d6c6d7a1383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0149f518784d4f6285de948b08383cca', 'request_id': 'req-feea2507-bcc1-456f-bde5-3d6c6d7a1383'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:52.768095 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-feea2507-bcc1-456f-bde5-3d6c6d7a1383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0149f518784d4f6285de948b08383cca', 'request_id': 'req-feea2507-bcc1-456f-bde5-3d6c6d7a1383'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:52.779436 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-feea2507-bcc1-456f-bde5-3d6c6d7a1383 admin admin] The token cache is being invalidate because project 0149f518784d4f6285de948b08383cca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:27:52.779727 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-feea2507-bcc1-456f-bde5-3d6c6d7a1383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:52.780639 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-feea2507-bcc1-456f-bde5-3d6c6d7a1383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:27:52.782205 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1415/2864] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:27:52 2022] DELETE /identity/v3/projects/0149f518784d4f6285de948b08383cca => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:52.787535 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee3d6f-d011-4bcb-813f-f61f6da708bb None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:52.788279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee3d6f-d011-4bcb-813f-f61f6da708bb None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:52.788279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee3d6f-d011-4bcb-813f-f61f6da708bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:52.806452 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6fee3d6f-d011-4bcb-813f-f61f6da708bb None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:52.817660 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6fee3d6f-d011-4bcb-813f-f61f6da708bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:52.866069 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1450/2865] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:27:52.875481 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6a775-b7f5-40f4-8006-c65832bad278 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:52.875801 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6a775-b7f5-40f4-8006-c65832bad278 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:52.876085 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6a775-b7f5-40f4-8006-c65832bad278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:52.894761 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d8f6a775-b7f5-40f4-8006-c65832bad278 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:52.895977 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d8f6a775-b7f5-40f4-8006-c65832bad278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:52.904305 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1416/2866] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:27:52.908685 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee3d6f-d011-4bcb-813f-f61f6da708bb None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:52.918958 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee3d6f-d011-4bcb-813f-f61f6da708bb None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:52.923598 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822bca0-3752-4e57-b31f-836fa99cac13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:52.924255 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822bca0-3752-4e57-b31f-836fa99cac13 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:52.924546 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822bca0-3752-4e57-b31f-836fa99cac13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:52.924828 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822bca0-3752-4e57-b31f-836fa99cac13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:52.925588 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822bca0-3752-4e57-b31f-836fa99cac13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:27:52.925899 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822bca0-3752-4e57-b31f-836fa99cac13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:52.927542 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822bca0-3752-4e57-b31f-836fa99cac13 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:52.939337 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1451/2867] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:27:52.943390 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6a775-b7f5-40f4-8006-c65832bad278 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:52.952920 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6a775-b7f5-40f4-8006-c65832bad278 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:52.957515 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09888cef-cd63-4e80-98ac-ba0be06b7787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:52.958152 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09888cef-cd63-4e80-98ac-ba0be06b7787 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:52.958435 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09888cef-cd63-4e80-98ac-ba0be06b7787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:52.958710 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09888cef-cd63-4e80-98ac-ba0be06b7787 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:52.959440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09888cef-cd63-4e80-98ac-ba0be06b7787 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:52.961332 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09888cef-cd63-4e80-98ac-ba0be06b7787 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:53.064731 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1417/2868] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:27:52 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:27:53.068793 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822bca0-3752-4e57-b31f-836fa99cac13 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:53.079335 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822bca0-3752-4e57-b31f-836fa99cac13 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:53.134928 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a8d0e-cd93-491c-b1cb-6356c45d9129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:53.135640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a8d0e-cd93-491c-b1cb-6356c45d9129 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:53.135931 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a8d0e-cd93-491c-b1cb-6356c45d9129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:53.136214 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a8d0e-cd93-491c-b1cb-6356c45d9129 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:53.136911 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a8d0e-cd93-491c-b1cb-6356c45d9129 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:53.138846 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a8d0e-cd93-491c-b1cb-6356c45d9129 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:53.251516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1452/2869] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:27:53 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:27:53.255570 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09888cef-cd63-4e80-98ac-ba0be06b7787 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:53.265958 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09888cef-cd63-4e80-98ac-ba0be06b7787 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:53.270601 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94903339-4e38-4272-a002-4eb45b5fd7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:53.273414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94903339-4e38-4272-a002-4eb45b5fd7f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:53.273414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94903339-4e38-4272-a002-4eb45b5fd7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:53.273414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94903339-4e38-4272-a002-4eb45b5fd7f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:53.273414 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94903339-4e38-4272-a002-4eb45b5fd7f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:53.274244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94903339-4e38-4272-a002-4eb45b5fd7f5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:53.284831 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1418/2870] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:27:53.288673 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a8d0e-cd93-491c-b1cb-6356c45d9129 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:53.299347 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a8d0e-cd93-491c-b1cb-6356c45d9129 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:53.304072 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247849c-4c51-4f3e-93c1-fdcbda43352e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:53.304726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247849c-4c51-4f3e-93c1-fdcbda43352e admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:53.305017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247849c-4c51-4f3e-93c1-fdcbda43352e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:53.305959 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247849c-4c51-4f3e-93c1-fdcbda43352e admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:53.306682 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247849c-4c51-4f3e-93c1-fdcbda43352e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:53.308379 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247849c-4c51-4f3e-93c1-fdcbda43352e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:53.318320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1453/2871] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:27:53.322341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94903339-4e38-4272-a002-4eb45b5fd7f5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:53.332374 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94903339-4e38-4272-a002-4eb45b5fd7f5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:53.337466 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7de782-a2d4-4de0-a994-54e3ff1471ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:53.341618 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7de782-a2d4-4de0-a994-54e3ff1471ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:53.341618 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7de782-a2d4-4de0-a994-54e3ff1471ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:53.341618 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7de782-a2d4-4de0-a994-54e3ff1471ee admin admin] PATH_INFO: `/v3/projects/742a40add8274bfb80151ac8dddddde3/users/6668064bd07b4a23a0de2a374c0ddaf4/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:53.353835 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7de782-a2d4-4de0-a994-54e3ff1471ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=742a40add8274bfb80151ac8dddddde3, user_id=6668064bd07b4a23a0de2a374c0ddaf4, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:53.356533 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7de782-a2d4-4de0-a994-54e3ff1471ee admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:53.423094 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1419/2872] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:27:53 2022] PUT /identity/v3/projects/742a40add8274bfb80151ac8dddddde3/users/6668064bd07b4a23a0de2a374c0ddaf4/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:53.433677 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22546852-5153-446a-82cb-f113e0657caf None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:53.433980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22546852-5153-446a-82cb-f113e0657caf None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:53.434267 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22546852-5153-446a-82cb-f113e0657caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:53.460097 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-22546852-5153-446a-82cb-f113e0657caf None None] MFA Rules not processed for user `6668064bd07b4a23a0de2a374c0ddaf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:53.461299 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-22546852-5153-446a-82cb-f113e0657caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:53.501987 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:53.523878 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1454/2873] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:27:53.528946 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7de782-a2d4-4de0-a994-54e3ff1471ee admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:53.573985 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:53.590979 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7de782-a2d4-4de0-a994-54e3ff1471ee admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:53.595697 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67a601-c6db-4d3c-b279-ba32085da0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:53.596352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67a601-c6db-4d3c-b279-ba32085da0f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:53.596640 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67a601-c6db-4d3c-b279-ba32085da0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:53.596919 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67a601-c6db-4d3c-b279-ba32085da0f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:53.597619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67a601-c6db-4d3c-b279-ba32085da0f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:53.599557 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67a601-c6db-4d3c-b279-ba32085da0f5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:53.667797 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1420/2874] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:27:53 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:27:53.671631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22546852-5153-446a-82cb-f113e0657caf None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:53.682193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22546852-5153-446a-82cb-f113e0657caf None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:53.738505 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c0a28-4126-4071-a083-69eaec0870c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:53.739197 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c0a28-4126-4071-a083-69eaec0870c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:53.739485 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c0a28-4126-4071-a083-69eaec0870c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:53.739761 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c0a28-4126-4071-a083-69eaec0870c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:53.740446 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c0a28-4126-4071-a083-69eaec0870c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:53.742361 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c0a28-4126-4071-a083-69eaec0870c7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:53.851951 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1455/2875] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:27:53 2022] POST /identity/v3/users => generated 299 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:27:53.855984 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67a601-c6db-4d3c-b279-ba32085da0f5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:53.866545 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67a601-c6db-4d3c-b279-ba32085da0f5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:53.871141 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f0e15-81d3-4fa5-b39e-646023b7a89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:53.871840 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f0e15-81d3-4fa5-b39e-646023b7a89a admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:53.872118 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f0e15-81d3-4fa5-b39e-646023b7a89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:53.872400 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f0e15-81d3-4fa5-b39e-646023b7a89a admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:53.873126 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f0e15-81d3-4fa5-b39e-646023b7a89a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:53.874752 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f0e15-81d3-4fa5-b39e-646023b7a89a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:53.885750 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1421/2876] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:27:53.889804 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c0a28-4126-4071-a083-69eaec0870c7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:53.900668 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c0a28-4126-4071-a083-69eaec0870c7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:53.905597 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b452eab-ff90-4d2c-bc5c-87f6d0647af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:53.906479 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b452eab-ff90-4d2c-bc5c-87f6d0647af8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:53.906765 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b452eab-ff90-4d2c-bc5c-87f6d0647af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:53.907046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b452eab-ff90-4d2c-bc5c-87f6d0647af8 admin admin] PATH_INFO: `/v3/projects/d4a4b4d84e284123a82ce099452fb6f7/users/504fc017fdbf4719a5c36426eb50a305/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:53.922617 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b452eab-ff90-4d2c-bc5c-87f6d0647af8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4a4b4d84e284123a82ce099452fb6f7, user_id=504fc017fdbf4719a5c36426eb50a305, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:53.925301 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b452eab-ff90-4d2c-bc5c-87f6d0647af8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:53.988395 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1456/2877] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:27:53 2022] PUT /identity/v3/projects/d4a4b4d84e284123a82ce099452fb6f7/users/504fc017fdbf4719a5c36426eb50a305/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:53.992408 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f0e15-81d3-4fa5-b39e-646023b7a89a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:54.002589 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f0e15-81d3-4fa5-b39e-646023b7a89a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:54.058551 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12bae6-693e-4166-8788-56f9bd642eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:54.059244 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12bae6-693e-4166-8788-56f9bd642eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:54.059540 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12bae6-693e-4166-8788-56f9bd642eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:54.059823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12bae6-693e-4166-8788-56f9bd642eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:54.060544 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12bae6-693e-4166-8788-56f9bd642eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:54.062147 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12bae6-693e-4166-8788-56f9bd642eb6 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:54.072650 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1422/2878] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:27:54.086877 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b452eab-ff90-4d2c-bc5c-87f6d0647af8 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:54.086877 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b452eab-ff90-4d2c-bc5c-87f6d0647af8 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:54.091525 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbfbcb-c52d-4834-a0c2-4943b35221f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:54.092398 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbfbcb-c52d-4834-a0c2-4943b35221f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:54.092684 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbfbcb-c52d-4834-a0c2-4943b35221f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:54.093021 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbfbcb-c52d-4834-a0c2-4943b35221f0 admin admin] PATH_INFO: `/v3/projects/d4a4b4d84e284123a82ce099452fb6f7/users/504fc017fdbf4719a5c36426eb50a305/roles/27fc14631c494ca0b2dc7f708be016c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:54.094982 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bbfbcb-c52d-4834-a0c2-4943b35221f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4a4b4d84e284123a82ce099452fb6f7, user_id=504fc017fdbf4719a5c36426eb50a305, role_id=27fc14631c494ca0b2dc7f708be016c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:54.097640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bbfbcb-c52d-4834-a0c2-4943b35221f0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:54.107843 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1457/2879] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:27:54 2022] PUT /identity/v3/projects/d4a4b4d84e284123a82ce099452fb6f7/users/504fc017fdbf4719a5c36426eb50a305/roles/27fc14631c494ca0b2dc7f708be016c2 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:27:54.124584 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61202c-c17c-43fe-a24e-6c35bbefe3aa None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:54.124909 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61202c-c17c-43fe-a24e-6c35bbefe3aa None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:54.125195 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61202c-c17c-43fe-a24e-6c35bbefe3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:54.152316 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4d61202c-c17c-43fe-a24e-6c35bbefe3aa None None] MFA Rules not processed for user `504fc017fdbf4719a5c36426eb50a305`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:27:54.153524 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4d61202c-c17c-43fe-a24e-6c35bbefe3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:54.223446 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1423/2880] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:27:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:27:54.233991 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbfbcb-c52d-4834-a0c2-4943b35221f0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:54.235726 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b5bbfbcb-c52d-4834-a0c2-4943b35221f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:54.293860 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbfbcb-c52d-4834-a0c2-4943b35221f0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:54.297440 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112267fd-213a-4b68-9dd8-68cc8d80963c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:54.298083 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112267fd-213a-4b68-9dd8-68cc8d80963c service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:54.298370 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112267fd-213a-4b68-9dd8-68cc8d80963c service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:54.298797 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112267fd-213a-4b68-9dd8-68cc8d80963c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:54.345360 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:27:54.355469 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112267fd-213a-4b68-9dd8-68cc8d80963c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:54.360937 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112267fd-213a-4b68-9dd8-68cc8d80963c service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:54.365727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1458/2881] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:27:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:27:59.629569 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61202c-c17c-43fe-a24e-6c35bbefe3aa None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:27:59.630996 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4d61202c-c17c-43fe-a24e-6c35bbefe3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:27:59.643341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61202c-c17c-43fe-a24e-6c35bbefe3aa None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:27:59.647373 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d6105-ec6e-4122-ab4b-845028303398 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:27:59.648336 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d6105-ec6e-4122-ab4b-845028303398 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:27:59.648619 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d6105-ec6e-4122-ab4b-845028303398 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:27:59.649462 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d6105-ec6e-4122-ab4b-845028303398 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:27:59.659827 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d6105-ec6e-4122-ab4b-845028303398 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:27:59.661270 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d6105-ec6e-4122-ab4b-845028303398 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:27:59.671539 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1424/2882] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:27:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:28:04.457358 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:28:05.318082 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112267fd-213a-4b68-9dd8-68cc8d80963c service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:05.380191 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112267fd-213a-4b68-9dd8-68cc8d80963c service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:05.384816 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0087d7-71d2-4095-93df-c4c5bd82600a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:05.386163 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0087d7-71d2-4095-93df-c4c5bd82600a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:05.386443 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0087d7-71d2-4095-93df-c4c5bd82600a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:05.386714 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0087d7-71d2-4095-93df-c4c5bd82600a admin admin] PATH_INFO: `/v3/users/6668064bd07b4a23a0de2a374c0ddaf4` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:05.387917 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0087d7-71d2-4095-93df-c4c5bd82600a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6668064bd07b4a23a0de2a374c0ddaf4)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:05.389835 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0087d7-71d2-4095-93df-c4c5bd82600a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:05.579466 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1a0087d7-71d2-4095-93df-c4c5bd82600a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6668064bd07b4a23a0de2a374c0ddaf4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:05.580132 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1a0087d7-71d2-4095-93df-c4c5bd82600a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6668064bd07b4a23a0de2a374c0ddaf4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:05.591813 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1a0087d7-71d2-4095-93df-c4c5bd82600a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6668064bd07b4a23a0de2a374c0ddaf4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:05.592447 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1a0087d7-71d2-4095-93df-c4c5bd82600a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6668064bd07b4a23a0de2a374c0ddaf4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:05.599387 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1a0087d7-71d2-4095-93df-c4c5bd82600a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6668064bd07b4a23a0de2a374c0ddaf4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:05.671236 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1459/2883] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:28:05 2022] DELETE /identity/v3/users/6668064bd07b4a23a0de2a374c0ddaf4 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:28:05.675733 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c1e21-6e6a-42dd-b71d-14605252d17b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:05.676221 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c1e21-6e6a-42dd-b71d-14605252d17b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:05.676334 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c1e21-6e6a-42dd-b71d-14605252d17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:05.694461 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-fd1c1e21-6e6a-42dd-b71d-14605252d17b None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:28:05.695711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-fd1c1e21-6e6a-42dd-b71d-14605252d17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:28:05.714957 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:28:05.756934 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1425/2884] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:28:05.766814 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0087d7-71d2-4095-93df-c4c5bd82600a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:05.768239 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1a0087d7-71d2-4095-93df-c4c5bd82600a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:28:05.825632 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0087d7-71d2-4095-93df-c4c5bd82600a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:05.829257 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d90098-9003-400a-9010-cd4e3f429048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:05.830019 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d90098-9003-400a-9010-cd4e3f429048 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:05.830247 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d90098-9003-400a-9010-cd4e3f429048 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:05.830526 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d90098-9003-400a-9010-cd4e3f429048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:05.839626 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d90098-9003-400a-9010-cd4e3f429048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:05.841071 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d90098-9003-400a-9010-cd4e3f429048 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:05.850498 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1460/2885] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:28:06.258886 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c1e21-6e6a-42dd-b71d-14605252d17b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:06.260330 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-fd1c1e21-6e6a-42dd-b71d-14605252d17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:28:06.274158 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c1e21-6e6a-42dd-b71d-14605252d17b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:06.279440 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c13a1f-6eb0-48d1-a770-14a05e34a3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:06.280620 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c13a1f-6eb0-48d1-a770-14a05e34a3f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:06.280906 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c13a1f-6eb0-48d1-a770-14a05e34a3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:06.281860 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c13a1f-6eb0-48d1-a770-14a05e34a3f0 admin admin] PATH_INFO: `/v3/projects/742a40add8274bfb80151ac8dddddde3` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:06.283308 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c13a1f-6eb0-48d1-a770-14a05e34a3f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=742a40add8274bfb80151ac8dddddde3)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:06.285311 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c13a1f-6eb0-48d1-a770-14a05e34a3f0 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:06.421384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-33c13a1f-6eb0-48d1-a770-14a05e34a3f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '742a40add8274bfb80151ac8dddddde3', 'request_id': 'req-33c13a1f-6eb0-48d1-a770-14a05e34a3f0'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:06.493425 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-33c13a1f-6eb0-48d1-a770-14a05e34a3f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '742a40add8274bfb80151ac8dddddde3', 'request_id': 'req-33c13a1f-6eb0-48d1-a770-14a05e34a3f0'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:06.499400 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-33c13a1f-6eb0-48d1-a770-14a05e34a3f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '742a40add8274bfb80151ac8dddddde3', 'request_id': 'req-33c13a1f-6eb0-48d1-a770-14a05e34a3f0'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:06.510593 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-33c13a1f-6eb0-48d1-a770-14a05e34a3f0 admin admin] The token cache is being invalidate because project 742a40add8274bfb80151ac8dddddde3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:28:06.511400 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-33c13a1f-6eb0-48d1-a770-14a05e34a3f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:06.512041 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-33c13a1f-6eb0-48d1-a770-14a05e34a3f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:06.513581 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1426/2886] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:28:06 2022] DELETE /identity/v3/projects/742a40add8274bfb80151ac8dddddde3 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:28:06.517597 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d90098-9003-400a-9010-cd4e3f429048 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:06.527808 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d90098-9003-400a-9010-cd4e3f429048 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:06.546099 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:28:06.583889 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed116926-a2f5-4d3c-a4a4-5de98c803b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:06.584762 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed116926-a2f5-4d3c-a4a4-5de98c803b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:06.585044 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed116926-a2f5-4d3c-a4a4-5de98c803b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:06.585322 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed116926-a2f5-4d3c-a4a4-5de98c803b08 admin admin] PATH_INFO: `/v3/users/504fc017fdbf4719a5c36426eb50a305` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:06.586589 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed116926-a2f5-4d3c-a4a4-5de98c803b08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=504fc017fdbf4719a5c36426eb50a305)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:06.588523 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed116926-a2f5-4d3c-a4a4-5de98c803b08 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:06.764610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ed116926-a2f5-4d3c-a4a4-5de98c803b08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '504fc017fdbf4719a5c36426eb50a305'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:06.765204 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ed116926-a2f5-4d3c-a4a4-5de98c803b08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '504fc017fdbf4719a5c36426eb50a305'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:06.776981 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ed116926-a2f5-4d3c-a4a4-5de98c803b08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '504fc017fdbf4719a5c36426eb50a305'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:06.777626 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ed116926-a2f5-4d3c-a4a4-5de98c803b08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '504fc017fdbf4719a5c36426eb50a305'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:06.784734 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ed116926-a2f5-4d3c-a4a4-5de98c803b08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '504fc017fdbf4719a5c36426eb50a305'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:06.853831 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1461/2887] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:28:06 2022] DELETE /identity/v3/users/504fc017fdbf4719a5c36426eb50a305 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:28:07.281541 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c13a1f-6eb0-48d1-a770-14a05e34a3f0 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:07.285102 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-33c13a1f-6eb0-48d1-a770-14a05e34a3f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:28:07.296820 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c13a1f-6eb0-48d1-a770-14a05e34a3f0 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:07.354137 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a298dfa-3835-4ae2-9fa5-065c00a38c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:07.355647 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a298dfa-3835-4ae2-9fa5-065c00a38c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:07.356000 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a298dfa-3835-4ae2-9fa5-065c00a38c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:07.356326 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a298dfa-3835-4ae2-9fa5-065c00a38c40 admin admin] PATH_INFO: `/v3/projects/d4a4b4d84e284123a82ce099452fb6f7` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:07.357605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a298dfa-3835-4ae2-9fa5-065c00a38c40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4a4b4d84e284123a82ce099452fb6f7)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:07.359539 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a298dfa-3835-4ae2-9fa5-065c00a38c40 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:07.453739 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9a298dfa-3835-4ae2-9fa5-065c00a38c40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4a4b4d84e284123a82ce099452fb6f7', 'request_id': 'req-9a298dfa-3835-4ae2-9fa5-065c00a38c40'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:07.455501 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:28:07.510762 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9a298dfa-3835-4ae2-9fa5-065c00a38c40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4a4b4d84e284123a82ce099452fb6f7', 'request_id': 'req-9a298dfa-3835-4ae2-9fa5-065c00a38c40'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:07.516609 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9a298dfa-3835-4ae2-9fa5-065c00a38c40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4a4b4d84e284123a82ce099452fb6f7', 'request_id': 'req-9a298dfa-3835-4ae2-9fa5-065c00a38c40'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:07.528187 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9a298dfa-3835-4ae2-9fa5-065c00a38c40 admin admin] The token cache is being invalidate because project d4a4b4d84e284123a82ce099452fb6f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:28:07.528424 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9a298dfa-3835-4ae2-9fa5-065c00a38c40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:07.529075 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9a298dfa-3835-4ae2-9fa5-065c00a38c40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:07.530969 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1427/2888] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:28:07 2022] DELETE /identity/v3/projects/d4a4b4d84e284123a82ce099452fb6f7 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:28:07.536909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6d7a3-8e33-4633-a1ed-11834c34b3b2 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:07.540713 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6d7a3-8e33-4633-a1ed-11834c34b3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:07.540713 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6d7a3-8e33-4633-a1ed-11834c34b3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:07.556080 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0de6d7a3-8e33-4633-a1ed-11834c34b3b2 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:28:07.557275 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0de6d7a3-8e33-4633-a1ed-11834c34b3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:28:07.614669 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1462/2889] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:28:07.623182 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cde33-67e4-44f4-92d9-6ca3b49db2d7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:07.623479 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cde33-67e4-44f4-92d9-6ca3b49db2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:07.623765 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cde33-67e4-44f4-92d9-6ca3b49db2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:07.641576 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-6a0cde33-67e4-44f4-92d9-6ca3b49db2d7 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:28:07.642780 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6a0cde33-67e4-44f4-92d9-6ca3b49db2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:28:07.650711 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1428/2890] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:28:07.655607 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6d7a3-8e33-4633-a1ed-11834c34b3b2 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:07.664960 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6d7a3-8e33-4633-a1ed-11834c34b3b2 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:07.670046 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866e29b-bd84-4f2d-bba6-c4cc1a0cb1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:07.670692 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866e29b-bd84-4f2d-bba6-c4cc1a0cb1ae admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:07.670980 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866e29b-bd84-4f2d-bba6-c4cc1a0cb1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:07.671294 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866e29b-bd84-4f2d-bba6-c4cc1a0cb1ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:07.672008 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866e29b-bd84-4f2d-bba6-c4cc1a0cb1ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:28:07.672316 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866e29b-bd84-4f2d-bba6-c4cc1a0cb1ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:07.673905 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866e29b-bd84-4f2d-bba6-c4cc1a0cb1ae admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:07.685306 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1463/2891] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:28:07.689520 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0cde33-67e4-44f4-92d9-6ca3b49db2d7 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:07.698691 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0cde33-67e4-44f4-92d9-6ca3b49db2d7 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:07.703399 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fcf63b-2889-4c09-bd12-8082767a0fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:07.704040 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcf63b-2889-4c09-bd12-8082767a0fc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:07.704326 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcf63b-2889-4c09-bd12-8082767a0fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:07.704605 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcf63b-2889-4c09-bd12-8082767a0fc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:07.705304 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fcf63b-2889-4c09-bd12-8082767a0fc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:07.707226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fcf63b-2889-4c09-bd12-8082767a0fc3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:07.791944 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1429/2892] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:28:07 2022] POST /identity/v3/projects => generated 359 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 19:28:07.797033 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866e29b-bd84-4f2d-bba6-c4cc1a0cb1ae admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:07.806912 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866e29b-bd84-4f2d-bba6-c4cc1a0cb1ae admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:07.832305 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:28:07.864424 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17ee7c-8a09-4cf7-8834-e5606153b075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:07.865124 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17ee7c-8a09-4cf7-8834-e5606153b075 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:07.865532 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17ee7c-8a09-4cf7-8834-e5606153b075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:07.865819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17ee7c-8a09-4cf7-8834-e5606153b075 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:07.866365 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17ee7c-8a09-4cf7-8834-e5606153b075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:07.868320 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17ee7c-8a09-4cf7-8834-e5606153b075 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:07.956190 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1464/2893] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:28:07 2022] POST /identity/v3/users => generated 280 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:28:07.960318 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fcf63b-2889-4c09-bd12-8082767a0fc3 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:07.971690 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fcf63b-2889-4c09-bd12-8082767a0fc3 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:07.976426 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530b4ef-d6f8-43f2-bacf-95a115e81b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:07.977090 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530b4ef-d6f8-43f2-bacf-95a115e81b46 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:07.977382 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530b4ef-d6f8-43f2-bacf-95a115e81b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:07.977665 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530b4ef-d6f8-43f2-bacf-95a115e81b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:07.978492 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530b4ef-d6f8-43f2-bacf-95a115e81b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:07.980138 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530b4ef-d6f8-43f2-bacf-95a115e81b46 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:07.990893 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1430/2894] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:28:07.994811 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17ee7c-8a09-4cf7-8834-e5606153b075 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:08.004994 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17ee7c-8a09-4cf7-8834-e5606153b075 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:08.009538 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488de68-549a-4342-9661-7b839f70ab89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:08.010193 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488de68-549a-4342-9661-7b839f70ab89 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:08.010474 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488de68-549a-4342-9661-7b839f70ab89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:08.010751 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488de68-549a-4342-9661-7b839f70ab89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:08.012137 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488de68-549a-4342-9661-7b839f70ab89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:08.013727 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488de68-549a-4342-9661-7b839f70ab89 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:08.023938 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1465/2895] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:28:08.028628 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530b4ef-d6f8-43f2-bacf-95a115e81b46 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:08.039254 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530b4ef-d6f8-43f2-bacf-95a115e81b46 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:08.043967 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02d4d3-c8b2-4981-aa81-aeebb88ad1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:08.044999 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02d4d3-c8b2-4981-aa81-aeebb88ad1d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:08.045294 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02d4d3-c8b2-4981-aa81-aeebb88ad1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:08.045579 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02d4d3-c8b2-4981-aa81-aeebb88ad1d2 admin admin] PATH_INFO: `/v3/projects/6a0bfa8f699d44368951c63a76af1816/users/6d865a031bef4a9b91b13a682eae6dc5/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:08.060652 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc02d4d3-c8b2-4981-aa81-aeebb88ad1d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a0bfa8f699d44368951c63a76af1816, user_id=6d865a031bef4a9b91b13a682eae6dc5, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:08.063475 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc02d4d3-c8b2-4981-aa81-aeebb88ad1d2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:08.171591 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1431/2896] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:28:08 2022] PUT /identity/v3/projects/6a0bfa8f699d44368951c63a76af1816/users/6d865a031bef4a9b91b13a682eae6dc5/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:28:08.183079 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d1318-9f02-4e0a-8816-cd1e24bec47d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:08.183395 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d1318-9f02-4e0a-8816-cd1e24bec47d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:08.183831 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d1318-9f02-4e0a-8816-cd1e24bec47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:08.210679 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-459d1318-9f02-4e0a-8816-cd1e24bec47d None None] MFA Rules not processed for user `6d865a031bef4a9b91b13a682eae6dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:28:08.211925 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-459d1318-9f02-4e0a-8816-cd1e24bec47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:28:08.284035 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1466/2897] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:28:08 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:28:08.325193 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02d4d3-c8b2-4981-aa81-aeebb88ad1d2 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:08.326886 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-bc02d4d3-c8b2-4981-aa81-aeebb88ad1d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:28:08.382384 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02d4d3-c8b2-4981-aa81-aeebb88ad1d2 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:08.386669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8769cd1-67f3-4c44-b093-72c1e77b3b7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:08.387347 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8769cd1-67f3-4c44-b093-72c1e77b3b7b service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:08.387640 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8769cd1-67f3-4c44-b093-72c1e77b3b7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:08.387918 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8769cd1-67f3-4c44-b093-72c1e77b3b7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:08.396117 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8769cd1-67f3-4c44-b093-72c1e77b3b7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:08.397548 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8769cd1-67f3-4c44-b093-72c1e77b3b7b service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:08.406524 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1432/2898] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:28:08 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:28:08.420673 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d1318-9f02-4e0a-8816-cd1e24bec47d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:08.480520 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d1318-9f02-4e0a-8816-cd1e24bec47d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:08.484986 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108e9b3-9b06-4205-80ca-e1c7635c99b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:08.485871 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108e9b3-9b06-4205-80ca-e1c7635c99b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:08.486154 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108e9b3-9b06-4205-80ca-e1c7635c99b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:08.486426 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108e9b3-9b06-4205-80ca-e1c7635c99b6 admin admin] PATH_INFO: `/v3/users/6d865a031bef4a9b91b13a682eae6dc5` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:08.487630 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108e9b3-9b06-4205-80ca-e1c7635c99b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d865a031bef4a9b91b13a682eae6dc5)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:08.489540 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108e9b3-9b06-4205-80ca-e1c7635c99b6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:08.666973 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:28:08.673171 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d108e9b3-9b06-4205-80ca-e1c7635c99b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d865a031bef4a9b91b13a682eae6dc5'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:08.673819 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d108e9b3-9b06-4205-80ca-e1c7635c99b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d865a031bef4a9b91b13a682eae6dc5'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:08.685890 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d108e9b3-9b06-4205-80ca-e1c7635c99b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d865a031bef4a9b91b13a682eae6dc5'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:08.686512 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d108e9b3-9b06-4205-80ca-e1c7635c99b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d865a031bef4a9b91b13a682eae6dc5'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:08.693152 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d108e9b3-9b06-4205-80ca-e1c7635c99b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d865a031bef4a9b91b13a682eae6dc5'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:08.762036 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1467/2899] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:28:08 2022] DELETE /identity/v3/users/6d865a031bef4a9b91b13a682eae6dc5 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:28:08.768624 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19359e-f151-4076-be67-f53279e09b9f None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:08.768953 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19359e-f151-4076-be67-f53279e09b9f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:08.769221 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19359e-f151-4076-be67-f53279e09b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:08.786926 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3e19359e-f151-4076-be67-f53279e09b9f None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:28:08.788264 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3e19359e-f151-4076-be67-f53279e09b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:28:08.847562 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1433/2900] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:28:08.857528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108e9b3-9b06-4205-80ca-e1c7635c99b6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:08.858918 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d108e9b3-9b06-4205-80ca-e1c7635c99b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:28:08.915909 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108e9b3-9b06-4205-80ca-e1c7635c99b6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:08.920108 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a19a3-5189-4a40-9c45-1a1a2feb6f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:08.920758 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a19a3-5189-4a40-9c45-1a1a2feb6f99 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:08.921047 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a19a3-5189-4a40-9c45-1a1a2feb6f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:08.921338 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a19a3-5189-4a40-9c45-1a1a2feb6f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:08.930574 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a19a3-5189-4a40-9c45-1a1a2feb6f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:08.932236 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a19a3-5189-4a40-9c45-1a1a2feb6f99 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:08.942461 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1468/2901] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:28:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:28:09.362170 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19359e-f151-4076-be67-f53279e09b9f None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:09.364823 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3e19359e-f151-4076-be67-f53279e09b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:28:09.377271 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19359e-f151-4076-be67-f53279e09b9f None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:09.382464 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0fe2e-1b78-42da-841b-a861f58979ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:09.383502 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0fe2e-1b78-42da-841b-a861f58979ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:09.383846 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0fe2e-1b78-42da-841b-a861f58979ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:09.384169 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0fe2e-1b78-42da-841b-a861f58979ee admin admin] PATH_INFO: `/v3/projects/6a0bfa8f699d44368951c63a76af1816` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:09.385242 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b0fe2e-1b78-42da-841b-a861f58979ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a0bfa8f699d44368951c63a76af1816)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:09.387805 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b0fe2e-1b78-42da-841b-a861f58979ee admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:09.399494 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:28:09.475566 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a9b0fe2e-1b78-42da-841b-a861f58979ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a0bfa8f699d44368951c63a76af1816', 'request_id': 'req-a9b0fe2e-1b78-42da-841b-a861f58979ee'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:09.542625 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a9b0fe2e-1b78-42da-841b-a861f58979ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a0bfa8f699d44368951c63a76af1816', 'request_id': 'req-a9b0fe2e-1b78-42da-841b-a861f58979ee'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:09.548254 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a9b0fe2e-1b78-42da-841b-a861f58979ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a0bfa8f699d44368951c63a76af1816', 'request_id': 'req-a9b0fe2e-1b78-42da-841b-a861f58979ee'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:09.559226 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a9b0fe2e-1b78-42da-841b-a861f58979ee admin admin] The token cache is being invalidate because project 6a0bfa8f699d44368951c63a76af1816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:28:09.559520 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a9b0fe2e-1b78-42da-841b-a861f58979ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:09.560159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a9b0fe2e-1b78-42da-841b-a861f58979ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:09.561718 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1434/2902] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:28:09 2022] DELETE /identity/v3/projects/6a0bfa8f699d44368951c63a76af1816 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:28:09.566628 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b6e3a-bb5a-4e68-a0a4-1680f6abf7c4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:09.566920 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b6e3a-bb5a-4e68-a0a4-1680f6abf7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:09.567228 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b6e3a-bb5a-4e68-a0a4-1680f6abf7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:09.584778 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e02b6e3a-bb5a-4e68-a0a4-1680f6abf7c4 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:28:09.585977 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e02b6e3a-bb5a-4e68-a0a4-1680f6abf7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:28:09.643905 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1469/2903] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:28:09.653157 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e674e-2ace-439d-b408-847db05ce3b1 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:09.653454 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e674e-2ace-439d-b408-847db05ce3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:09.653759 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e674e-2ace-439d-b408-847db05ce3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:09.671959 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-cc6e674e-2ace-439d-b408-847db05ce3b1 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:28:09.673159 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-cc6e674e-2ace-439d-b408-847db05ce3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:28:09.681204 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1435/2904] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:28:09.685212 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b6e3a-bb5a-4e68-a0a4-1680f6abf7c4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:09.694657 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b6e3a-bb5a-4e68-a0a4-1680f6abf7c4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:09.699754 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d1b40-2b51-49f3-9011-e931b51a7fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:09.700404 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d1b40-2b51-49f3-9011-e931b51a7fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:09.700694 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d1b40-2b51-49f3-9011-e931b51a7fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:09.700974 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d1b40-2b51-49f3-9011-e931b51a7fd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:09.701693 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d1b40-2b51-49f3-9011-e931b51a7fd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 19:28:09.702004 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d1b40-2b51-49f3-9011-e931b51a7fd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:09.703640 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d1b40-2b51-49f3-9011-e931b51a7fd5 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:09.715556 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1470/2905] 10.222.0.69 () {58 vars in 1152 bytes} [Thu Jul 14 19:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 19:28:09.719664 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e674e-2ace-439d-b408-847db05ce3b1 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:09.728735 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e674e-2ace-439d-b408-847db05ce3b1 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:09.733386 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792cd7d-a72f-4b62-8e70-9d96a4aa2b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:09.734120 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792cd7d-a72f-4b62-8e70-9d96a4aa2b7e admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:09.734407 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792cd7d-a72f-4b62-8e70-9d96a4aa2b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:09.734683 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792cd7d-a72f-4b62-8e70-9d96a4aa2b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:09.735420 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792cd7d-a72f-4b62-8e70-9d96a4aa2b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:09.737310 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792cd7d-a72f-4b62-8e70-9d96a4aa2b7e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:09.847352 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1436/2906] 10.222.0.69 () {60 vars in 1152 bytes} [Thu Jul 14 19:28:09 2022] POST /identity/v3/projects => generated 399 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:28:09.851649 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d1b40-2b51-49f3-9011-e931b51a7fd5 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:09.861638 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d1b40-2b51-49f3-9011-e931b51a7fd5 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:09.906403 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:28:09.918939 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c734c-0fd6-443c-af18-9db0f6a92145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:09.919654 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c734c-0fd6-443c-af18-9db0f6a92145 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:09.919945 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c734c-0fd6-443c-af18-9db0f6a92145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:09.920222 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c734c-0fd6-443c-af18-9db0f6a92145 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:09.920905 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c734c-0fd6-443c-af18-9db0f6a92145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:09.922832 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c734c-0fd6-443c-af18-9db0f6a92145 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:10.017789 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1471/2907] 10.222.0.69 () {60 vars in 1143 bytes} [Thu Jul 14 19:28:09 2022] POST /identity/v3/users => generated 300 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 19:28:10.022045 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792cd7d-a72f-4b62-8e70-9d96a4aa2b7e admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:10.032451 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792cd7d-a72f-4b62-8e70-9d96a4aa2b7e admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:10.037072 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6433e-ca08-4733-abc1-555906b7aa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:10.037725 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6433e-ca08-4733-abc1-555906b7aa45 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:10.038008 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6433e-ca08-4733-abc1-555906b7aa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:10.038283 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6433e-ca08-4733-abc1-555906b7aa45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:10.039057 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6433e-ca08-4733-abc1-555906b7aa45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:10.040682 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6433e-ca08-4733-abc1-555906b7aa45 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:10.050838 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1437/2908] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:28:10.054717 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c734c-0fd6-443c-af18-9db0f6a92145 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:10.065017 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c734c-0fd6-443c-af18-9db0f6a92145 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:10.070432 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c63478-f7a2-4f75-a917-faddfbffeac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:10.071101 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c63478-f7a2-4f75-a917-faddfbffeac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:10.071520 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c63478-f7a2-4f75-a917-faddfbffeac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:10.071809 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c63478-f7a2-4f75-a917-faddfbffeac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:10.072579 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c63478-f7a2-4f75-a917-faddfbffeac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:10.074838 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c63478-f7a2-4f75-a917-faddfbffeac1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:10.102687 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1472/2909] 10.222.0.69 () {58 vars in 1121 bytes} [Thu Jul 14 19:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 19:28:10.107485 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6433e-ca08-4733-abc1-555906b7aa45 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:10.117508 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6433e-ca08-4733-abc1-555906b7aa45 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:10.122775 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288221ca-571e-477a-a2f6-61aa2770d80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:10.123669 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288221ca-571e-477a-a2f6-61aa2770d80a admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:10.123952 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288221ca-571e-477a-a2f6-61aa2770d80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:10.124230 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288221ca-571e-477a-a2f6-61aa2770d80a admin admin] PATH_INFO: `/v3/projects/48d3854a888848a7b25752a720ecbbf5/users/370fbf8e7fea49589c7fb4ff6afb6615/roles/ffed4493824a4040977d2cd1d057e07b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:10.138447 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288221ca-571e-477a-a2f6-61aa2770d80a admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d3854a888848a7b25752a720ecbbf5, user_id=370fbf8e7fea49589c7fb4ff6afb6615, role_id=ffed4493824a4040977d2cd1d057e07b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:10.141208 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288221ca-571e-477a-a2f6-61aa2770d80a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:10.224751 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1438/2910] 10.222.0.69 () {60 vars in 1482 bytes} [Thu Jul 14 19:28:10 2022] PUT /identity/v3/projects/48d3854a888848a7b25752a720ecbbf5/users/370fbf8e7fea49589c7fb4ff6afb6615/roles/ffed4493824a4040977d2cd1d057e07b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:28:10.235720 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ddf05-0ced-4923-a33a-7820a9d6feac None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:10.236064 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ddf05-0ced-4923-a33a-7820a9d6feac None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:10.236543 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ddf05-0ced-4923-a33a-7820a9d6feac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:10.273610 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d08ddf05-0ced-4923-a33a-7820a9d6feac None None] MFA Rules not processed for user `370fbf8e7fea49589c7fb4ff6afb6615`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:28:10.274832 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d08ddf05-0ced-4923-a33a-7820a9d6feac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:28:10.338886 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1473/2911] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:28:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 19:28:10.350316 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288221ca-571e-477a-a2f6-61aa2770d80a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:10.352036 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-288221ca-571e-477a-a2f6-61aa2770d80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:28:10.410119 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288221ca-571e-477a-a2f6-61aa2770d80a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:10.416148 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591863d-3ec5-4deb-89a4-2562254d2fed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b09d72314349c6be0f906484895608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:10.417785 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591863d-3ec5-4deb-89a4-2562254d2fed service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:10.418272 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591863d-3ec5-4deb-89a4-2562254d2fed service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:10.418771 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591863d-3ec5-4deb-89a4-2562254d2fed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:10.428471 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591863d-3ec5-4deb-89a4-2562254d2fed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:10.429922 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591863d-3ec5-4deb-89a4-2562254d2fed service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:10.439447 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1439/2912] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:28:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:28:19.279157 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ddf05-0ced-4923-a33a-7820a9d6feac None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:19.341024 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ddf05-0ced-4923-a33a-7820a9d6feac None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:19.345565 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12580343-6173-48a0-9248-864cc90480b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:19.346439 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12580343-6173-48a0-9248-864cc90480b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:19.346729 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12580343-6173-48a0-9248-864cc90480b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:19.347012 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12580343-6173-48a0-9248-864cc90480b0 admin admin] PATH_INFO: `/v3/users/370fbf8e7fea49589c7fb4ff6afb6615` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:19.348238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12580343-6173-48a0-9248-864cc90480b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=370fbf8e7fea49589c7fb4ff6afb6615)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:19.350177 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12580343-6173-48a0-9248-864cc90480b0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:19.521815 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-12580343-6173-48a0-9248-864cc90480b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '370fbf8e7fea49589c7fb4ff6afb6615'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:19.522422 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-12580343-6173-48a0-9248-864cc90480b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '370fbf8e7fea49589c7fb4ff6afb6615'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:19.530196 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:28:19.535600 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-12580343-6173-48a0-9248-864cc90480b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '370fbf8e7fea49589c7fb4ff6afb6615'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:19.536238 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-12580343-6173-48a0-9248-864cc90480b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '370fbf8e7fea49589c7fb4ff6afb6615'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:19.543558 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-12580343-6173-48a0-9248-864cc90480b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '370fbf8e7fea49589c7fb4ff6afb6615'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:19.604856 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1474/2913] 10.222.0.69 () {58 vars in 1223 bytes} [Thu Jul 14 19:28:19 2022] DELETE /identity/v3/users/370fbf8e7fea49589c7fb4ff6afb6615 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:28:19.610038 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6125c-a8f6-4126-a5f6-50d8f1b4cbe4 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:19.610518 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6125c-a8f6-4126-a5f6-50d8f1b4cbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:19.610637 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6125c-a8f6-4126-a5f6-50d8f1b4cbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:19.627824 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4bd6125c-a8f6-4126-a5f6-50d8f1b4cbe4 None None] MFA Rules not processed for user `82583283d3254dca9292c9de336ad74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 19:28:19.629020 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4bd6125c-a8f6-4126-a5f6-50d8f1b4cbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:28:19.662341 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:28:19.687810 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1440/2914] 10.222.0.69 () {58 vars in 957 bytes} [Thu Jul 14 19:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 19:28:19.697528 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12580343-6173-48a0-9248-864cc90480b0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:19.698929 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-12580343-6173-48a0-9248-864cc90480b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:28:19.756818 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12580343-6173-48a0-9248-864cc90480b0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:19.760631 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b45fb5-ee46-48a9-94f6-e542f42a1914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74b8c75b9cce442b8c430fab6a3fea17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:19.761279 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b45fb5-ee46-48a9-94f6-e542f42a1914 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:19.761570 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b45fb5-ee46-48a9-94f6-e542f42a1914 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:19.761852 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b45fb5-ee46-48a9-94f6-e542f42a1914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:19.770803 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b45fb5-ee46-48a9-94f6-e542f42a1914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:19.772330 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b45fb5-ee46-48a9-94f6-e542f42a1914 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:19.782516 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1475/2915] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:28:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 19:28:20.183345 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6125c-a8f6-4126-a5f6-50d8f1b4cbe4 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:28:20.183345 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4bd6125c-a8f6-4126-a5f6-50d8f1b4cbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:28:20.195070 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6125c-a8f6-4126-a5f6-50d8f1b4cbe4 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:28:20.201853 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc51d3-39f9-48a5-962d-e16f4a98ed4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82583283d3254dca9292c9de336ad74a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ecc6a6cebab48d4b8cfff094ca5066b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:28:20.202845 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc51d3-39f9-48a5-962d-e16f4a98ed4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:28:20.208688 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc51d3-39f9-48a5-962d-e16f4a98ed4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:28:20.211815 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc51d3-39f9-48a5-962d-e16f4a98ed4c admin admin] PATH_INFO: `/v3/projects/48d3854a888848a7b25752a720ecbbf5` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:28:20.211815 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc51d3-39f9-48a5-962d-e16f4a98ed4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=48d3854a888848a7b25752a720ecbbf5)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:28:20.213021 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc51d3-39f9-48a5-962d-e16f4a98ed4c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:28:20.312003 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-71bc51d3-39f9-48a5-962d-e16f4a98ed4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48d3854a888848a7b25752a720ecbbf5', 'request_id': 'req-71bc51d3-39f9-48a5-962d-e16f4a98ed4c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:20.384872 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-71bc51d3-39f9-48a5-962d-e16f4a98ed4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48d3854a888848a7b25752a720ecbbf5', 'request_id': 'req-71bc51d3-39f9-48a5-962d-e16f4a98ed4c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:20.391269 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-71bc51d3-39f9-48a5-962d-e16f4a98ed4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48d3854a888848a7b25752a720ecbbf5', 'request_id': 'req-71bc51d3-39f9-48a5-962d-e16f4a98ed4c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:20.402574 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-71bc51d3-39f9-48a5-962d-e16f4a98ed4c admin admin] The token cache is being invalidate because project 48d3854a888848a7b25752a720ecbbf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 19:28:20.402864 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-71bc51d3-39f9-48a5-962d-e16f4a98ed4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:20.403561 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-71bc51d3-39f9-48a5-962d-e16f4a98ed4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 19:28:20.405106 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1441/2916] 10.222.0.69 () {58 vars in 1232 bytes} [Thu Jul 14 19:28:20 2022] DELETE /identity/v3/projects/48d3854a888848a7b25752a720ecbbf5 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 19:28:29.808648 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=35,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:28:30.399359 dv-cinder-799871-31-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 19:32:10.075149 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b45fb5-ee46-48a9-94f6-e542f42a1914 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 19:32:10.076550 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-96b45fb5-ee46-48a9-94f6-e542f42a1914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:32:10.144197 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b45fb5-ee46-48a9-94f6-e542f42a1914 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 19:32:10.202931 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa1867-a2a3-44e2-82e8-075934107492 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c577fa8670e34449a80de0c8e0a15a8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6003dba8ddd247e7842d28ea7ca5238c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 19:32:10.203618 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa1867-a2a3-44e2-82e8-075934107492 service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 19:32:10.203921 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa1867-a2a3-44e2-82e8-075934107492 service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 19:32:10.204197 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa1867-a2a3-44e2-82e8-075934107492 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 19:32:10.205963 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a1fa1867-a2a3-44e2-82e8-075934107492 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 19:32:10.217571 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa1867-a2a3-44e2-82e8-075934107492 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 19:32:10.219141 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa1867-a2a3-44e2-82e8-075934107492 service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 19:32:10.228889 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1476/2917] 10.222.0.69 () {60 vars in 1365 bytes} [Thu Jul 14 19:32:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 19:32:20.224599 dv-cinder-799871-31-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=58 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}